-- Logs begin at Thu 2022-09-08 00:33:37 UTC, end at Thu 2022-09-08 01:30:41 UTC. -- Sep 08 00:58:49.813944 dv-cinder-847733-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 08 00:58:49.821285 dv-cinder-847733-3-iscsi devstack@keystone.service[81646]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 08 00:58:49.823340 dv-cinder-847733-3-iscsi devstack@keystone.service[81646]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Sep 8 00:58:49 2022] *** Sep 08 00:58:49.823340 dv-cinder-847733-3-iscsi devstack@keystone.service[81646]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 08 00:58:49.823340 dv-cinder-847733-3-iscsi devstack@keystone.service[81646]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 08 00:58:49.823340 dv-cinder-847733-3-iscsi devstack@keystone.service[81646]: nodename: dv-cinder-847733-3-iscsi Sep 08 00:58:49.823340 dv-cinder-847733-3-iscsi devstack@keystone.service[81646]: machine: x86_64 Sep 08 00:58:49.823340 dv-cinder-847733-3-iscsi devstack@keystone.service[81646]: clock source: unix Sep 08 00:58:49.823340 dv-cinder-847733-3-iscsi devstack@keystone.service[81646]: pcre jit disabled Sep 08 00:58:49.823785 dv-cinder-847733-3-iscsi devstack@keystone.service[81646]: detected number of CPU cores: 4 Sep 08 00:58:49.823785 dv-cinder-847733-3-iscsi devstack@keystone.service[81646]: current working directory: / Sep 08 00:58:49.823785 dv-cinder-847733-3-iscsi devstack@keystone.service[81646]: detected binary path: /usr/bin/uwsgi-core Sep 08 00:58:49.823785 dv-cinder-847733-3-iscsi devstack@keystone.service[81646]: your processes number limit is 47680 Sep 08 00:58:49.823785 dv-cinder-847733-3-iscsi devstack@keystone.service[81646]: your memory page size is 4096 bytes Sep 08 00:58:49.823785 dv-cinder-847733-3-iscsi devstack@keystone.service[81646]: detected max file descriptor number: 2048 Sep 08 00:58:49.823785 dv-cinder-847733-3-iscsi devstack@keystone.service[81646]: lock engine: pthread robust mutexes Sep 08 00:58:49.823785 dv-cinder-847733-3-iscsi devstack@keystone.service[81646]: thunder lock: enabled Sep 08 00:58:49.823785 dv-cinder-847733-3-iscsi devstack@keystone.service[81646]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 08 00:58:49.823785 dv-cinder-847733-3-iscsi devstack@keystone.service[81646]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 08 00:58:49.866441 dv-cinder-847733-3-iscsi devstack@keystone.service[81646]: Python main interpreter initialized at 0x557ccb30a9c0 Sep 08 00:58:49.866441 dv-cinder-847733-3-iscsi devstack@keystone.service[81646]: python threads support enabled Sep 08 00:58:49.866441 dv-cinder-847733-3-iscsi devstack@keystone.service[81646]: your server socket listen backlog is limited to 100 connections Sep 08 00:58:49.866441 dv-cinder-847733-3-iscsi devstack@keystone.service[81646]: your mercy for graceful operations on workers is 90 seconds Sep 08 00:58:49.866841 dv-cinder-847733-3-iscsi devstack@keystone.service[81646]: mapped 403077 bytes (393 KB) for 2 cores Sep 08 00:58:49.866924 dv-cinder-847733-3-iscsi devstack@keystone.service[81646]: *** Operational MODE: preforking *** Sep 08 00:58:49.867187 dv-cinder-847733-3-iscsi devstack@keystone.service[81646]: *** uWSGI is running in multiple interpreter mode *** Sep 08 00:58:49.867187 dv-cinder-847733-3-iscsi devstack@keystone.service[81646]: spawned uWSGI master process (pid: 81646) Sep 08 00:58:49.867391 dv-cinder-847733-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 08 00:58:49.868168 dv-cinder-847733-3-iscsi devstack@keystone.service[81646]: spawned uWSGI worker 1 (pid: 81660, cores: 1) Sep 08 00:58:49.870207 dv-cinder-847733-3-iscsi devstack@keystone.service[81646]: spawned uWSGI worker 2 (pid: 81661, cores: 1) Sep 08 00:58:49.870207 dv-cinder-847733-3-iscsi devstack@keystone.service[81646]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 08 00:58:54.177214 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 08 00:58:54.177214 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: from cryptography.utils import int_from_bytes Sep 08 00:58:54.178009 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 08 00:58:54.178009 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: from cryptography.utils import int_from_bytes Sep 08 00:58:54.178455 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 08 00:58:54.178455 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: from cryptography.utils import int_from_bytes Sep 08 00:58:54.178589 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 08 00:58:54.178589 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: from cryptography.utils import int_from_bytes Sep 08 00:58:54.227613 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 08 00:58:54.227815 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 08 00:58:54.227961 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 08 00:58:54.228058 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] command line args: [] {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 08 00:58:54.228168 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 08 00:58:54.228288 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 08 00:58:54.228382 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] command line args: [] {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 08 00:58:54.228462 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ================================================================================ {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 08 00:58:54.228618 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 08 00:58:54.228704 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] admin_token = **** {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.228800 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ================================================================================ {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 08 00:58:54.228932 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] config_dir = [] {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.229056 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] admin_token = **** {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.229141 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.229283 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] config_source = [] {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.229360 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] config_dir = [] {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.229462 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] debug = True {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.229547 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.229630 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] config_source = [] {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.229706 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.229951 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] debug = True {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.230061 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.230196 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.230196 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.230382 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.230382 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.230561 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.230662 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.230769 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.230871 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] list_limit = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.230971 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.231076 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] log_config_append = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.231170 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] list_limit = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.231262 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.231370 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] log_config_append = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.231466 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] log_dir = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.231572 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.231673 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] log_file = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.231767 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] log_dir = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.231874 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.231999 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] log_file = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.232081 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.232231 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.232348 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.232422 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.232543 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.232633 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.232705 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.232821 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.232896 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.233055 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.233178 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.233257 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.233348 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.233457 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.233672 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.233771 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.233935 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.234019 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.234168 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.234289 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.234429 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.234536 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.234657 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.234794 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.234922 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.235061 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.235174 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.235316 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.235435 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.235601 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.235782 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] publish_errors = False {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.235892 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.236041 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] publish_errors = False {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.236183 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.236302 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.236416 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.236535 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.236641 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.236641 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.236770 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.236894 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.236977 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.237091 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] standard_threads = False {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.237175 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.237321 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.237401 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] standard_threads = False {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.237552 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.237629 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.237724 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.237800 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.237902 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] use_journal = False {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.238009 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.238132 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] use_json = False {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.238212 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] use_journal = False {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.238282 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] use_stderr = False {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.238357 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] use_json = False {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.238430 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] use_syslog = False {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.238499 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] use_stderr = False {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.238606 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] use_syslog = False {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.238684 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.238837 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 00:58:54.238914 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.239075 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.239151 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.239322 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.239404 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.239555 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.239638 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.239802 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.239881 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.240037 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.240115 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.240300 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.240384 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.240532 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.240609 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.240773 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.240851 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.241016 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.241091 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.241262 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.241344 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.241492 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.241576 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.241731 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.241808 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.241961 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.242070 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.242202 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.242280 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.242446 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.242523 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.242678 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.242764 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.242914 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.242997 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.243170 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.243247 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.243409 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.243562 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.243642 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.243712 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.243882 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.243966 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.244159 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.244251 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.244416 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.244494 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.244662 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.244738 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.244894 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.244980 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.245125 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.245202 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.245356 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.245434 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.245585 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.245673 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.245818 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.245895 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.246067 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.246144 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.246296 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.246382 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.246525 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.246602 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.246753 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.246828 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.247002 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.247088 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.247236 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.247312 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.247475 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.247550 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.247727 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.247918 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.247997 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.248174 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.248257 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.248415 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.248492 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.249148 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.249235 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.249388 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.249472 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.249641 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.249723 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.249874 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.249987 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.250097 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.250227 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.250346 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.250482 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.250568 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.250725 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.250850 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.250960 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.251043 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.251193 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.251284 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.251429 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.251509 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.251666 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.251745 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.251903 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.251984 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.252149 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.252226 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.252979 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.253069 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.253069 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.253188 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.253252 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.253316 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.253419 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.253496 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.253576 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.253650 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.253717 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.253775 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.253831 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.253895 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.253961 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.254026 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.254170 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.254257 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.254419 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.254513 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.254652 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.254740 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.254881 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.254965 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.255118 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.255212 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.255350 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.255431 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.255587 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.255668 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.255840 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.255926 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.256086 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.256241 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.256409 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.256585 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.256771 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.256951 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.257139 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.257327 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.257517 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.257662 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.257772 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.257881 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.257988 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.258087 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.258179 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.258254 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.258358 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.258433 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.258503 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.258572 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.258651 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.258761 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.258841 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.258920 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.258992 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.259063 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.259144 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.259218 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.259288 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.259358 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.259493 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.259574 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.260008 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.260094 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.260272 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.260354 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.260522 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.260602 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.260779 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.260867 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.260996 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.261075 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.261231 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.261308 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.261456 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.261531 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.261684 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.261762 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.261916 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.262003 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.262144 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.262232 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.262374 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.262451 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.262599 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.262682 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.262828 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.262917 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.263065 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.263145 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.263296 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.263426 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.263540 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.263668 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.263749 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.263920 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.264008 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.264178 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.264345 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.265056 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.265253 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.265340 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.265554 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.265650 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.265790 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] auth.external = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.265884 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.266021 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.266110 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] auth.external = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.266292 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.266370 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.266540 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.266635 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.266771 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] auth.password = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.266865 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.266999 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] auth.token = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.267079 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] auth.password = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.267237 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.267316 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] auth.token = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.267471 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.267558 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.267717 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.267797 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.267948 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.268036 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.268223 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.268319 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.268471 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.268551 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.268702 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.268780 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.268936 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] credential.caching = True {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.269027 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.269175 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] credential.caching = True {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.269254 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.269418 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.269496 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.269667 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.269754 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.269905 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.269984 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.270149 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.270228 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.270382 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.270469 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.270622 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.270709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.270873 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.270953 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.271107 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.271197 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.271359 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.271438 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.271617 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.271696 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.271872 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.271951 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.272156 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.272240 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.272416 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.272516 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.272662 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.272748 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.272923 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.273002 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] federation.caching = True {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.273160 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] federation.caching = True {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.273239 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.273401 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.273482 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.273653 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.273732 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.273888 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.273966 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.274124 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.274204 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.274357 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.274434 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.274601 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.274678 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.274834 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.274913 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.275084 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.275162 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.275316 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.275393 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.275562 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.275644 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.275795 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] identity.caching = True {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.275874 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.276032 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] identity.caching = True {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.276109 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.276311 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.276389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.276522 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.276603 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.276755 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.276834 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.277000 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.277081 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.277228 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.277308 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.277461 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.277540 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.277709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.277790 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.277959 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.278040 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.278195 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.278274 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.278425 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.278506 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.278657 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.278737 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.278891 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.278972 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.279135 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.279214 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.279379 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.279459 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.279626 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.279705 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.279871 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.279999 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.280191 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.280353 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.280472 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.280675 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.280766 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.280890 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.280970 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.281122 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.281201 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.281360 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.281441 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.281590 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.281668 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.281823 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.281902 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.282063 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.282144 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.282302 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.282381 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.282548 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.282626 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.282779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.282859 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.283024 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.283102 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.283269 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.283347 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.283501 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.283581 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.283745 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.283823 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.283991 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.284069 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.284243 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.284336 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.284482 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.284562 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.284713 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.284791 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.284946 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.285026 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.285181 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.285261 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.285419 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.285497 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.285653 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.285735 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.286004 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.286082 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.286258 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.286338 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.286509 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.286599 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.286742 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.286821 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.286972 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.287050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.287223 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.287303 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.287470 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.287548 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.287705 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.287783 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.287939 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.288019 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.288200 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.288283 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.288432 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ldap.user = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.288510 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ldap.user = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.288671 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.288752 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.288927 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.289006 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.289160 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.289238 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.289406 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.289486 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.289652 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.289731 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.289898 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.289976 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.290131 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.290212 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.290361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.290439 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.290592 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.290670 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.290825 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.290905 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.291057 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.291135 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.291286 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.291364 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.291529 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.291610 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.291774 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.291853 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.292019 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.292097 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.292285 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.292367 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.292532 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.292610 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.292758 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.292836 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.293004 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.293084 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.293236 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.293314 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.293469 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.293548 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.293704 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.293784 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.293934 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.294012 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.294177 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.294255 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.294421 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.294502 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.294657 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.294736 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.294908 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.294986 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.295137 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.295217 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.295375 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.295454 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.295707 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.295791 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.295863 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.295924 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.296081 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.296191 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.296350 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.296443 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.296592 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.296673 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.296821 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.296900 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.297053 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.297142 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.297292 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] resource.caching = True {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.297371 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] resource.caching = True {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.297538 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.297618 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.297784 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.297872 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.298017 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.298096 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.298264 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.298342 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.298510 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.298599 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.298744 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.298824 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.298988 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.299067 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.299222 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.299377 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.299454 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.299535 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.299679 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] role.caching = True {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.299759 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] role.caching = True {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.299921 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] role.driver = sql {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.300009 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] role.driver = sql {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.300171 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.300255 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.300409 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.300487 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.300654 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.300901 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.300988 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.301145 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.301226 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.301390 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.301469 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.301635 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.301717 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.301880 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.301958 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.302123 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.302201 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.302353 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.302434 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.302590 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.302667 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.302832 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.302911 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.303077 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.303157 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.303317 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.303396 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.303562 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.303716 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.303954 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.304033 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.304214 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.304308 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.304466 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.304555 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.304713 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.304792 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.304960 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.305038 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.305192 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.305279 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.305431 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.305523 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.305682 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.305782 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.305918 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.306012 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.306147 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.306227 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.306377 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.306457 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.306603 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.306690 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.306836 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.306915 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.307086 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.307166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.307328 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.307414 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.307562 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.307642 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.307795 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.307874 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.308029 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.308116 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.308281 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.308360 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] token.caching = True {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.308514 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.308592 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] token.caching = True {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.308748 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.308839 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.308982 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.309061 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.309223 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.309340 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.309473 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.309559 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.309708 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.309788 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.309941 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.310020 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.310183 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.310277 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.310418 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.310497 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.310650 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.310728 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.310886 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.310974 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.311119 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.311747 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.311896 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.312078 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.312188 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.312379 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.312554 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.312642 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.312801 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.312882 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.313068 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.313163 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.313325 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] database.connection = **** {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.313406 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.313566 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.313648 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] database.connection = **** {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.313812 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.313891 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.314046 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.314132 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.314286 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.314368 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.314518 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.314597 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.314752 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.314830 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.314982 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.315062 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.315215 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.315296 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.315453 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.315532 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.315684 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.315764 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.315924 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.316002 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.316169 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.316252 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.316420 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.316504 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.316655 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.316734 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.316891 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.316969 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.317119 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.317200 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.317356 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.317436 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.317587 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.317675 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.317750 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81661) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 08 00:58:54.317822 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 00:58:54.317977 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81660) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 08 00:58:54.347431 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81661) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.347657 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81661) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.347881 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81661) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.347997 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81660) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.348266 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81660) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.348497 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81660) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.356232 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81661) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.356863 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81660) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.385146 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81661) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.385371 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81661) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.385488 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81660) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.385720 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81660) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.439284 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81661) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.439622 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81660) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.488047 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81661) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.488301 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81661) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.488557 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81660) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.488793 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81660) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.646042 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81661) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.646170 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81660) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.646283 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81661) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.646406 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81660) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.646515 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81661) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.646631 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81660) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.646717 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81661) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.646850 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81660) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.646933 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81661) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.647067 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81660) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.647153 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81661) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.647283 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81660) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.670097 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81661) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.670313 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81660) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.670400 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81661) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.671057 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81661) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.671151 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81660) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.671283 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81661) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.671367 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81660) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.671502 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81661) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.671596 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81660) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.671720 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81660) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.671804 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81661) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.671937 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81660) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.672021 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81661) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.672168 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81660) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.683140 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81661) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.683456 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81660) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.695865 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81661) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.696075 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81661) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.696267 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81660) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.696354 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81661) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.696504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81661) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.696587 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81660) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.696721 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81661) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.696804 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81660) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.696936 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81661) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.697021 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81660) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.697151 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81660) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.697233 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81661) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.697369 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81660) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.697585 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81660) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 00:58:54.714375 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.714587 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 08 00:58:54.714735 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.714929 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.715013 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 08 00:58:54.715130 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 08 00:58:54.715284 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.715401 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.715483 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 08 00:58:54.715660 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.715750 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.715919 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.716001 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.716202 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.716288 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.716521 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.725886 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.726163 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.726433 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.726621 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.726704 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.726902 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.727174 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.727460 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.734398 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81661) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 00:58:54.735028 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81660) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 00:58:54.737318 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81661) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 00:58:54.737604 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.737888 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.737971 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81660) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 00:58:54.738170 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.738254 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.738454 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.738539 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.738715 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.738798 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.738985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.739078 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.739260 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.739342 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.739530 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.739610 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.739799 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.739890 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.740065 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.740170 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.740364 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.740641 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.763127 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.763469 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.765138 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81661) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 00:58:54.765441 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.765592 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81660) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 00:58:54.765878 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.770353 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81661) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 00:58:54.770572 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81660) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 00:58:54.770687 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.770859 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.770953 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.771150 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.777192 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81661) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 00:58:54.777476 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.777558 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81660) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 00:58:54.777779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.783228 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81661) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 00:58:54.783458 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81660) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 00:58:54.783546 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.783758 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.783841 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.784052 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.784160 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.784364 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.784446 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.784643 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.784739 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.784916 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.784998 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.785184 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.785265 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.785457 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.801040 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.801317 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.801775 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.802056 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.805126 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81661) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 00:58:54.806190 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81660) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 00:58:54.808297 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.808593 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.809456 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.809754 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.812781 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81661) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 00:58:54.814169 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81660) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 00:58:54.815887 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81661) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 00:58:54.817321 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81660) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 00:58:54.819108 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.819405 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.819691 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.819974 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.820290 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.820557 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.820658 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.820862 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.821152 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.821439 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.821726 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.822019 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.837505 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81661) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 00:58:54.837788 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.838053 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.838317 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.838533 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81660) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 00:58:54.838822 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.839091 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.839354 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.845273 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.846431 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.846796 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.847977 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.848341 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.848617 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.849537 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.849825 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.851315 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81661) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 00:58:54.851600 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.851879 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.852195 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.852495 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81660) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 00:58:54.852589 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.852783 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.853066 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.853350 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.853749 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.864235 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81661) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 00:58:54.864516 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.864796 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.865075 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.865455 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.865538 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81660) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 00:58:54.865744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.865828 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.866023 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.866106 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.866361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.866746 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.867029 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.867320 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.881414 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81661) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 00:58:54.882938 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81660) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 00:58:54.884298 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81661) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 00:58:54.886149 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81660) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 00:58:54.887421 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.888947 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.889124 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.890491 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81661) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 00:58:54.890770 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.890856 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.891161 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.892262 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81660) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 00:58:54.892555 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.892950 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.897509 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.898959 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81661) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 00:58:54.899296 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.900781 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81660) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 00:58:54.901853 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.902250 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.902537 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.902817 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.903675 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.904076 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.904390 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.904671 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.911212 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81661) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 00:58:54.911502 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.911782 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.913382 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81660) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 00:58:54.913674 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.913952 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.918628 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81661) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 00:58:54.918913 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.919194 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.919469 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.919753 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.920027 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.920327 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.920659 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.920948 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.921039 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81660) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 00:58:54.921268 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.921406 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.921553 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.921783 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.921871 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.922056 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.922145 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.922323 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.922404 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.922596 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81661) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.922677 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.922917 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.923195 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.923600 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.923943 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.924239 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.924513 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.924794 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81660) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 00:58:54.957335 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x557ccb30a9c0 pid: 81661 (default app) Sep 08 00:58:54.959089 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x557ccb30a9c0 pid: 81660 (default app) Sep 08 00:58:54.960557 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b7a29-72b9-4329-bac0-f884c053192d None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:58:54.960844 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b7a29-72b9-4329-bac0-f884c053192d None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:58:54.961114 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b7a29-72b9-4329-bac0-f884c053192d None None] PATH_INFO: `/v3` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:58:54.965200 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1/1] 10.222.0.42 () {50 vars in 791 bytes} [Thu Sep 8 00:58:54 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:01.752265 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e86e2-eb5e-4494-bc94-17bf92239cd7 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:01.752796 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e86e2-eb5e-4494-bc94-17bf92239cd7 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:01.752796 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e86e2-eb5e-4494-bc94-17bf92239cd7 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:01.757463 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1/2] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:01 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 00:59:01.763158 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d8907-50c3-4198-8d5f-c6440d599a80 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:01.763647 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d8907-50c3-4198-8d5f-c6440d599a80 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:01.763738 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d8907-50c3-4198-8d5f-c6440d599a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:01.835655 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: INFO dbcounter [None req-ec6d8907-50c3-4198-8d5f-c6440d599a80 None None] Registered counter for database keystone Sep 08 00:59:01.854599 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG oslo_db.sqlalchemy.engines [None req-ec6d8907-50c3-4198-8d5f-c6440d599a80 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81661) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 08 00:59:01.861104 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writer thread running {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 08 00:59:02.042557 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG passlib.handlers.bcrypt [None req-ec6d8907-50c3-4198-8d5f-c6440d599a80 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81661) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 08 00:59:02.042955 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG passlib.handlers.bcrypt [None req-ec6d8907-50c3-4198-8d5f-c6440d599a80 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81661) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 08 00:59:02.101250 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-ec6d8907-50c3-4198-8d5f-c6440d599a80 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:02.102864 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-ec6d8907-50c3-4198-8d5f-c6440d599a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:02.235251 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 2/3] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:01 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 473 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 08 00:59:02.539700 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e86e2-eb5e-4494-bc94-17bf92239cd7 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:02.611764 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: INFO dbcounter [None req-981e86e2-eb5e-4494-bc94-17bf92239cd7 None None] Registered counter for database keystone Sep 08 00:59:02.632956 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG oslo_db.sqlalchemy.engines [None req-981e86e2-eb5e-4494-bc94-17bf92239cd7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81660) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 08 00:59:02.639534 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writer thread running {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 08 00:59:02.759280 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e86e2-eb5e-4494-bc94-17bf92239cd7 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:02.763832 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7817687a-38f7-47fa-83bd-f859fdd6f711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:02.765041 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7817687a-38f7-47fa-83bd-f859fdd6f711 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:02.765320 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7817687a-38f7-47fa-83bd-f859fdd6f711 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:02.765590 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7817687a-38f7-47fa-83bd-f859fdd6f711 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:02.773841 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7817687a-38f7-47fa-83bd-f859fdd6f711 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:02.824886 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: WARNING keystone.common.rbac_enforcer.enforcer [None req-7817687a-38f7-47fa-83bd-f859fdd6f711 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 08 00:59:02.825190 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7817687a-38f7-47fa-83bd-f859fdd6f711 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:02.835597 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 2/4] 10.222.0.42 () {58 vars in 1324 bytes} [Thu Sep 8 00:59:02 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 00:59:02.841517 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6d8907-50c3-4198-8d5f-c6440d599a80 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:02.859342 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6d8907-50c3-4198-8d5f-c6440d599a80 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:02.863724 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae27604-368c-4b6a-835f-cdff997ad658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:02.864867 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae27604-368c-4b6a-835f-cdff997ad658 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:02.865144 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae27604-368c-4b6a-835f-cdff997ad658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:02.865409 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae27604-368c-4b6a-835f-cdff997ad658 admin admin] PATH_INFO: `/v3/projects/fc3cae79b39a4ab893f53897bb099478` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:02.866991 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae27604-368c-4b6a-835f-cdff997ad658 admin admin] RBAC: Authorizing `identity:get_project(project_id=fc3cae79b39a4ab893f53897bb099478)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:02.919968 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: WARNING keystone.common.rbac_enforcer.enforcer [None req-4ae27604-368c-4b6a-835f-cdff997ad658 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 08 00:59:02.920970 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae27604-368c-4b6a-835f-cdff997ad658 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:02.925555 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 3/5] 10.222.0.42 () {56 vars in 1207 bytes} [Thu Sep 8 00:59:02 2022] GET /identity/v3/projects/fc3cae79b39a4ab893f53897bb099478 => generated 344 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 00:59:05.038851 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad168dc-ef2b-47a8-be5e-1743fe71701a None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:05.039398 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad168dc-ef2b-47a8-be5e-1743fe71701a None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:05.039398 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad168dc-ef2b-47a8-be5e-1743fe71701a None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:05.043528 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 3/6] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 00:59:05.049058 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ede9cc-7330-4c75-a27c-bb9c06ad3969 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:05.049468 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ede9cc-7330-4c75-a27c-bb9c06ad3969 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:05.049607 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ede9cc-7330-4c75-a27c-bb9c06ad3969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:05.069474 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-19ede9cc-7330-4c75-a27c-bb9c06ad3969 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:05.070981 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-19ede9cc-7330-4c75-a27c-bb9c06ad3969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:05.079393 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 4/7] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:05 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 08 00:59:05.343361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad168dc-ef2b-47a8-be5e-1743fe71701a None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:05.355350 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad168dc-ef2b-47a8-be5e-1743fe71701a None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:05.360197 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e46f56-d2df-41a6-bd4e-7ef1ae282d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:05.360792 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e46f56-d2df-41a6-bd4e-7ef1ae282d99 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:05.361072 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e46f56-d2df-41a6-bd4e-7ef1ae282d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:05.361342 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e46f56-d2df-41a6-bd4e-7ef1ae282d99 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:05.368924 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e46f56-d2df-41a6-bd4e-7ef1ae282d99 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:05.370695 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e46f56-d2df-41a6-bd4e-7ef1ae282d99 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:05.380101 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 4/8] 10.222.0.42 () {58 vars in 1324 bytes} [Thu Sep 8 00:59:05 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 00:59:05.385555 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ede9cc-7330-4c75-a27c-bb9c06ad3969 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:05.394786 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ede9cc-7330-4c75-a27c-bb9c06ad3969 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:05.399170 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffcead8-1652-4507-832a-0023be96ee13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:05.399795 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffcead8-1652-4507-832a-0023be96ee13 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:05.400084 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffcead8-1652-4507-832a-0023be96ee13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:05.400408 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffcead8-1652-4507-832a-0023be96ee13 admin admin] PATH_INFO: `/v3/users/51f97eb5e7894117a81b5bd04a11488e` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:05.401728 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffcead8-1652-4507-832a-0023be96ee13 admin admin] RBAC: Authorizing `identity:get_user(user_id=51f97eb5e7894117a81b5bd04a11488e)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:05.403550 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffcead8-1652-4507-832a-0023be96ee13 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:05.405602 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 5/9] 10.222.0.42 () {56 vars in 1198 bytes} [Thu Sep 8 00:59:05 2022] GET /identity/v3/users/51f97eb5e7894117a81b5bd04a11488e => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:07.617865 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be90949d-4d25-47cb-8476-b680fbd2753a None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:07.618395 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be90949d-4d25-47cb-8476-b680fbd2753a None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:07.618395 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be90949d-4d25-47cb-8476-b680fbd2753a None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:07.619976 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 5/10] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 00:59:07.625241 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baeb6fb-94cd-4855-bb6e-90645fd1f7bf None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:07.625645 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baeb6fb-94cd-4855-bb6e-90645fd1f7bf None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:07.625791 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baeb6fb-94cd-4855-bb6e-90645fd1f7bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:07.644154 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-4baeb6fb-94cd-4855-bb6e-90645fd1f7bf None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:07.645333 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-4baeb6fb-94cd-4855-bb6e-90645fd1f7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:07.681657 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bad355-b648-4c60-b73b-f4030fe0eef9 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:07.681934 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bad355-b648-4c60-b73b-f4030fe0eef9 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:07.682209 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bad355-b648-4c60-b73b-f4030fe0eef9 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:07.683757 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 6/11] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 00:59:07.688900 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a077fa17-a4b4-4b15-a08e-0c4166c08759 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:07.689187 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a077fa17-a4b4-4b15-a08e-0c4166c08759 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:07.689462 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a077fa17-a4b4-4b15-a08e-0c4166c08759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:07.727535 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 6/12] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 00:59:07.731774 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG passlib.handlers.bcrypt [None req-a077fa17-a4b4-4b15-a08e-0c4166c08759 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81660) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 08 00:59:07.732196 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG passlib.handlers.bcrypt [None req-a077fa17-a4b4-4b15-a08e-0c4166c08759 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81660) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 08 00:59:07.782317 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-a077fa17-a4b4-4b15-a08e-0c4166c08759 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:07.783898 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-a077fa17-a4b4-4b15-a08e-0c4166c08759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:07.884226 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 7/13] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 00:59:08.004290 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baeb6fb-94cd-4855-bb6e-90645fd1f7bf None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:08.028152 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:08.107693 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baeb6fb-94cd-4855-bb6e-90645fd1f7bf None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:08.142034 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b688ae-8c33-425b-8994-749619599e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:08.142606 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b688ae-8c33-425b-8994-749619599e72 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:08.142889 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b688ae-8c33-425b-8994-749619599e72 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:08.143166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b688ae-8c33-425b-8994-749619599e72 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:08.154164 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b688ae-8c33-425b-8994-749619599e72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 00:59:08.161816 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a077fa17-a4b4-4b15-a08e-0c4166c08759 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:08.163161 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-f2b688ae-8c33-425b-8994-749619599e72 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 00:59:08.163161 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 00:59:08.163161 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 00:59:08.163161 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 00:59:08.163161 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 00:59:08.163161 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 00:59:08.163161 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 00:59:08.163161 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 00:59:08.163161 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 00:59:08.163161 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 00:59:08.163161 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 00:59:08.163161 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 00:59:08.163161 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 00:59:08.163161 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 00:59:08.163161 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 00:59:08.163161 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 00:59:08.164125 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 00:59:08.164125 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 00:59:08.164125 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 00:59:08.164125 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 00:59:08.164125 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 00:59:08.164125 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 00:59:08.164125 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 00:59:08.164125 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 00:59:08.164125 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 00:59:08.164125 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 00:59:08.164125 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 00:59:08.164125 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 00:59:08.164125 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 00:59:08.164125 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 00:59:08.164125 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 00:59:08.164125 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 00:59:08.165063 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 00:59:08.165063 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 00:59:08.165063 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 00:59:08.165063 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 00:59:08.165063 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 00:59:08.165063 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 00:59:08.165063 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 00:59:08.165063 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 00:59:08.165063 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 00:59:08.165063 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 00:59:08.165063 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 00:59:08.165063 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 00:59:08.165063 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 7/14] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 8 00:59:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 00:59:08.169893 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b688ae-8c33-425b-8994-749619599e72 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:08.252260 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:08.265410 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a077fa17-a4b4-4b15-a08e-0c4166c08759 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:08.267883 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:08.271287 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b688ae-8c33-425b-8994-749619599e72 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:08.304487 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f33fe0-38e6-4abe-889c-41223e683d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:08.304949 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3895d2-db00-4f55-b318-a5ebc09f5511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:08.305348 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f33fe0-38e6-4abe-889c-41223e683d5e None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:08.305554 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3895d2-db00-4f55-b318-a5ebc09f5511 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:08.305664 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f33fe0-38e6-4abe-889c-41223e683d5e None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:08.305836 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3895d2-db00-4f55-b318-a5ebc09f5511 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:08.305932 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f33fe0-38e6-4abe-889c-41223e683d5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:08.306110 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3895d2-db00-4f55-b318-a5ebc09f5511 None admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:08.306776 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3895d2-db00-4f55-b318-a5ebc09f5511 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:08.307082 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3895d2-db00-4f55-b318-a5ebc09f5511 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:08.308508 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3895d2-db00-4f55-b318-a5ebc09f5511 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:08.320757 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 8/15] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 00:59:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 00:59:08.326200 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3895d2-db00-4f55-b318-a5ebc09f5511 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:08.375301 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f33fe0-38e6-4abe-889c-41223e683d5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:08.376878 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f33fe0-38e6-4abe-889c-41223e683d5e None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:08.424766 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3895d2-db00-4f55-b318-a5ebc09f5511 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:08.458187 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cfae0a-c916-462a-9bfc-0881a1473097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:08.458818 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cfae0a-c916-462a-9bfc-0881a1473097 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:08.459099 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cfae0a-c916-462a-9bfc-0881a1473097 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:08.459379 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cfae0a-c916-462a-9bfc-0881a1473097 None admin] PATH_INFO: `/v3/users/51f97eb5e7894117a81b5bd04a11488e` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:08.460544 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cfae0a-c916-462a-9bfc-0881a1473097 None admin] RBAC: Authorizing `identity:get_user(user_id=51f97eb5e7894117a81b5bd04a11488e)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:08.462019 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cfae0a-c916-462a-9bfc-0881a1473097 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:08.463576 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 9/16] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 8 00:59:08 2022] GET /identity/v3/users/51f97eb5e7894117a81b5bd04a11488e => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 00:59:08.469711 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cfae0a-c916-462a-9bfc-0881a1473097 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:08.474942 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 8/17] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 00:59:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 08 00:59:08.480373 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f33fe0-38e6-4abe-889c-41223e683d5e None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:08.483822 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:08.484472 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:08.570410 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cfae0a-c916-462a-9bfc-0881a1473097 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:08.578643 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f33fe0-38e6-4abe-889c-41223e683d5e None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:08.606141 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf36a9e-3218-49d1-a950-a5e179dd8ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:08.606765 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf36a9e-3218-49d1-a950-a5e179dd8ee0 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:08.607041 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf36a9e-3218-49d1-a950-a5e179dd8ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:08.607312 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf36a9e-3218-49d1-a950-a5e179dd8ee0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:08.609333 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf36a9e-3218-49d1-a950-a5e179dd8ee0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:08.610780 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf36a9e-3218-49d1-a950-a5e179dd8ee0 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:08.612365 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 10/18] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 00:59:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:08.617729 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47ad5d6-fda6-4792-93a6-f537e2a9549f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:08.618004 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf36a9e-3218-49d1-a950-a5e179dd8ee0 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:08.618610 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ad5d6-fda6-4792-93a6-f537e2a9549f None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:08.619036 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ad5d6-fda6-4792-93a6-f537e2a9549f None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:08.619210 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ad5d6-fda6-4792-93a6-f537e2a9549f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:08.639241 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47ad5d6-fda6-4792-93a6-f537e2a9549f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:08.640774 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47ad5d6-fda6-4792-93a6-f537e2a9549f None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:08.659366 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-f47ad5d6-fda6-4792-93a6-f537e2a9549f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 00:59:08.659366 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 00:59:08.659366 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 00:59:08.659366 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 00:59:08.659366 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 00:59:08.659366 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 00:59:08.659366 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 00:59:08.659366 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 00:59:08.659366 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 00:59:08.659366 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 00:59:08.659366 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 00:59:08.659366 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 00:59:08.659366 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 00:59:08.659366 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 00:59:08.659366 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 00:59:08.659366 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 00:59:08.660296 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 00:59:08.660296 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 00:59:08.660296 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 00:59:08.660296 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 00:59:08.660296 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 00:59:08.660296 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 00:59:08.660296 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 00:59:08.660296 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 00:59:08.660296 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 00:59:08.660296 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 00:59:08.660296 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 00:59:08.660296 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 00:59:08.660296 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 00:59:08.660296 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 00:59:08.660296 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 00:59:08.660296 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 00:59:08.660296 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 00:59:08.661284 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 00:59:08.661284 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 00:59:08.661284 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 00:59:08.661284 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 9/19] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 00:59:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 00:59:08.666397 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47ad5d6-fda6-4792-93a6-f537e2a9549f None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:08.701924 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:08.721748 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf36a9e-3218-49d1-a950-a5e179dd8ee0 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:08.722066 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:08.756497 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0a3fd6-e689-499d-a980-83955fd23c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:08.757107 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a3fd6-e689-499d-a980-83955fd23c55 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:08.757389 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a3fd6-e689-499d-a980-83955fd23c55 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:08.757662 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a3fd6-e689-499d-a980-83955fd23c55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:08.758387 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0a3fd6-e689-499d-a980-83955fd23c55 None admin] RBAC: Adding query filter params (role.id=34de369ec7c54715aa83a9052a26c3f2, scope.domain.id=default, user.id=51f97eb5e7894117a81b5bd04a11488e) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:08.758699 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0a3fd6-e689-499d-a980-83955fd23c55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:08.760422 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0a3fd6-e689-499d-a980-83955fd23c55 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:08.767583 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 11/20] 10.222.0.42 () {56 vars in 1322 bytes} [Thu Sep 8 00:59:08 2022] GET /identity/v3/role_assignments?user.id=51f97eb5e7894117a81b5bd04a11488e&scope.domain.id=default&role.id=34de369ec7c54715aa83a9052a26c3f2 => generated 233 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 00:59:08.768057 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47ad5d6-fda6-4792-93a6-f537e2a9549f None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:08.804024 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b384a76-2925-4588-84dc-dafa26bcdf49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:08.804669 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b384a76-2925-4588-84dc-dafa26bcdf49 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:08.804952 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b384a76-2925-4588-84dc-dafa26bcdf49 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:08.805225 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b384a76-2925-4588-84dc-dafa26bcdf49 None admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:08.805890 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b384a76-2925-4588-84dc-dafa26bcdf49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:08.806196 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b384a76-2925-4588-84dc-dafa26bcdf49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:08.807634 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b384a76-2925-4588-84dc-dafa26bcdf49 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:08.827987 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 10/21] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 00:59:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 00:59:11.294353 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733bcbe-1959-4af0-ba8f-493a2922dd3e None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:11.294878 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733bcbe-1959-4af0-ba8f-493a2922dd3e None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:11.295685 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733bcbe-1959-4af0-ba8f-493a2922dd3e None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:11.297370 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 12/22] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 00:59:11.302921 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0adc8eb-e0cc-4df7-b2ae-06cce61b417a None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:11.303426 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0adc8eb-e0cc-4df7-b2ae-06cce61b417a None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:11.303500 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0adc8eb-e0cc-4df7-b2ae-06cce61b417a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:11.326087 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-e0adc8eb-e0cc-4df7-b2ae-06cce61b417a None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:11.327925 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-e0adc8eb-e0cc-4df7-b2ae-06cce61b417a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:11.414296 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 11/23] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 00:59:11.751106 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee5fab-5bf1-45c8-bb88-de3596c30d7d None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:11.751386 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee5fab-5bf1-45c8-bb88-de3596c30d7d None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:11.751816 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee5fab-5bf1-45c8-bb88-de3596c30d7d None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:11.753603 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 13/24] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 00:59:11.758427 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd054fdc-4a47-425b-97bc-5d4354cd0d2d None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:11.758926 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd054fdc-4a47-425b-97bc-5d4354cd0d2d None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:11.759356 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd054fdc-4a47-425b-97bc-5d4354cd0d2d None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:11.761468 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 12/25] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 00:59:11.765380 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ceb3a6-f65e-472a-94fc-a0174cf899e5 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:11.765827 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ceb3a6-f65e-472a-94fc-a0174cf899e5 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:11.766280 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ceb3a6-f65e-472a-94fc-a0174cf899e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:11.771730 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970e917-2eaa-429a-8c1a-33e6ecf32431 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:11.775945 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970e917-2eaa-429a-8c1a-33e6ecf32431 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:11.776474 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970e917-2eaa-429a-8c1a-33e6ecf32431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:11.793976 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-a4ceb3a6-f65e-472a-94fc-a0174cf899e5 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:11.796333 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-a4ceb3a6-f65e-472a-94fc-a0174cf899e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:11.823220 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-c970e917-2eaa-429a-8c1a-33e6ecf32431 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:11.828727 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-c970e917-2eaa-429a-8c1a-33e6ecf32431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:11.929778 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 14/26] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 00:59:11.960373 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:12.003072 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 13/27] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 00:59:12.046454 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ceb3a6-f65e-472a-94fc-a0174cf899e5 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:12.066301 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264274b-db45-4d59-af75-4093fed84728 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:12.066689 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264274b-db45-4d59-af75-4093fed84728 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:12.066830 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264274b-db45-4d59-af75-4093fed84728 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:12.068657 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 14/28] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 00:59:12.074166 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df44ffcf-85c5-466b-8ca9-605b5127c95d None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:12.074451 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df44ffcf-85c5-466b-8ca9-605b5127c95d None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:12.074727 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df44ffcf-85c5-466b-8ca9-605b5127c95d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:12.103666 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-df44ffcf-85c5-466b-8ca9-605b5127c95d None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:12.106843 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-df44ffcf-85c5-466b-8ca9-605b5127c95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:12.253983 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:12.307482 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 15/29] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 00:59:12.308358 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df44ffcf-85c5-466b-8ca9-605b5127c95d None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:12.331732 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ceb3a6-f65e-472a-94fc-a0174cf899e5 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:12.405725 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0261f8a-ac17-4441-8bb4-3e9c33e553ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:12.407032 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0261f8a-ac17-4441-8bb4-3e9c33e553ad None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:12.407500 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0261f8a-ac17-4441-8bb4-3e9c33e553ad None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:12.407946 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0261f8a-ac17-4441-8bb4-3e9c33e553ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:12.435568 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0261f8a-ac17-4441-8bb4-3e9c33e553ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 00:59:12.447451 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-a0261f8a-ac17-4441-8bb4-3e9c33e553ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 00:59:12.447451 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 00:59:12.447451 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 00:59:12.447451 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 00:59:12.447451 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 00:59:12.447451 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 00:59:12.447451 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 00:59:12.447451 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 00:59:12.447451 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 00:59:12.447451 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 00:59:12.447451 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 00:59:12.447451 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 00:59:12.447451 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 00:59:12.447451 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 00:59:12.447451 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 00:59:12.447451 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 00:59:12.448403 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 00:59:12.448403 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 00:59:12.448403 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 00:59:12.448403 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 00:59:12.448403 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 00:59:12.448403 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 00:59:12.448403 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 00:59:12.448403 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 00:59:12.448403 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 00:59:12.448403 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 00:59:12.448403 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 00:59:12.448403 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 00:59:12.448403 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 00:59:12.448403 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 00:59:12.448403 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 00:59:12.448403 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 00:59:12.449172 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 00:59:12.449172 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 00:59:12.449172 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 00:59:12.449172 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 00:59:12.449172 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 00:59:12.449172 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 00:59:12.449172 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 00:59:12.449172 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 00:59:12.449172 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 00:59:12.449172 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 00:59:12.449172 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 00:59:12.449172 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 00:59:12.456413 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 15/30] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 8 00:59:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 410 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 00:59:12.457731 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0261f8a-ac17-4441-8bb4-3e9c33e553ad None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:12.522027 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df44ffcf-85c5-466b-8ca9-605b5127c95d None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:12.531586 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:12.566136 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dafbfa-044b-4d6e-8026-0218d9227245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:12.567226 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dafbfa-044b-4d6e-8026-0218d9227245 None admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:12.567787 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dafbfa-044b-4d6e-8026-0218d9227245 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:12.568300 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dafbfa-044b-4d6e-8026-0218d9227245 None admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:12.569830 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dafbfa-044b-4d6e-8026-0218d9227245 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:12.571613 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dafbfa-044b-4d6e-8026-0218d9227245 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:12.688053 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0261f8a-ac17-4441-8bb4-3e9c33e553ad None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:12.722103 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 16/31] 10.222.0.42 () {60 vars in 1131 bytes} [Thu Sep 8 00:59:12 2022] POST /identity/v3/roles => generated 219 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 00:59:12.723623 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dafbfa-044b-4d6e-8026-0218d9227245 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:12.726079 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209e606d-3c03-409c-8afe-c815dd1be45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:12.726741 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e606d-3c03-409c-8afe-c815dd1be45d None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:12.727073 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e606d-3c03-409c-8afe-c815dd1be45d None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:12.727355 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e606d-3c03-409c-8afe-c815dd1be45d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:12.730728 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:12.743543 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209e606d-3c03-409c-8afe-c815dd1be45d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:12.745289 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209e606d-3c03-409c-8afe-c815dd1be45d None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:12.763725 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-209e606d-3c03-409c-8afe-c815dd1be45d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 00:59:12.763725 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 00:59:12.763725 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 00:59:12.763725 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 00:59:12.763725 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 00:59:12.763725 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 00:59:12.763725 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 00:59:12.763725 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 00:59:12.763725 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 00:59:12.763725 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 00:59:12.763725 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 00:59:12.763725 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 00:59:12.763725 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 00:59:12.763725 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 00:59:12.763725 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 00:59:12.763725 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 00:59:12.764729 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 00:59:12.764729 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 00:59:12.764729 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 00:59:12.764729 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 00:59:12.764729 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 00:59:12.764729 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 00:59:12.764729 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 00:59:12.764729 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 00:59:12.764729 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 00:59:12.764729 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 00:59:12.764729 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 00:59:12.764729 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 00:59:12.764729 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 00:59:12.764729 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 00:59:12.764729 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 00:59:12.764729 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 00:59:12.764729 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 00:59:12.765506 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 00:59:12.765506 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 00:59:12.765506 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 00:59:12.765506 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 16/32] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 00:59:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 00:59:12.771337 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209e606d-3c03-409c-8afe-c815dd1be45d None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:12.864334 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dafbfa-044b-4d6e-8026-0218d9227245 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:12.916028 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9296a9e-80f7-46fe-a3a2-5bee6245b778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:12.917057 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9296a9e-80f7-46fe-a3a2-5bee6245b778 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:12.919027 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9296a9e-80f7-46fe-a3a2-5bee6245b778 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:12.919528 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9296a9e-80f7-46fe-a3a2-5bee6245b778 None admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:12.921395 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9296a9e-80f7-46fe-a3a2-5bee6245b778 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:12.923527 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9296a9e-80f7-46fe-a3a2-5bee6245b778 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:12.925463 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9296a9e-80f7-46fe-a3a2-5bee6245b778 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:12.949093 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 17/33] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 00:59:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:12.950648 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9296a9e-80f7-46fe-a3a2-5bee6245b778 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:12.970050 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209e606d-3c03-409c-8afe-c815dd1be45d None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:12.987205 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:13.028441 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee77d19-2cf9-4a1b-9c65-c1416fe95ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:13.029174 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee77d19-2cf9-4a1b-9c65-c1416fe95ced None admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:13.030066 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee77d19-2cf9-4a1b-9c65-c1416fe95ced None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:13.030474 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee77d19-2cf9-4a1b-9c65-c1416fe95ced None admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:13.031402 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee77d19-2cf9-4a1b-9c65-c1416fe95ced None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:13.033511 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee77d19-2cf9-4a1b-9c65-c1416fe95ced None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:13.085489 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9296a9e-80f7-46fe-a3a2-5bee6245b778 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:13.121740 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a75aab-d5dd-400a-b299-24d6958ec0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:13.122631 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a75aab-d5dd-400a-b299-24d6958ec0b4 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:13.123082 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a75aab-d5dd-400a-b299-24d6958ec0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:13.124292 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a75aab-d5dd-400a-b299-24d6958ec0b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:13.125182 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a75aab-d5dd-400a-b299-24d6958ec0b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:13.127713 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a75aab-d5dd-400a-b299-24d6958ec0b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:13.129523 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a75aab-d5dd-400a-b299-24d6958ec0b4 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:13.143944 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 18/34] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 00:59:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:13.145091 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a75aab-d5dd-400a-b299-24d6958ec0b4 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:13.175943 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 17/35] 10.222.0.42 () {60 vars in 1131 bytes} [Thu Sep 8 00:59:12 2022] POST /identity/v3/roles => generated 225 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 00:59:13.177542 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3bd91b-bdab-4e99-9137-b5f15b7dcf31 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:13.177835 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3bd91b-bdab-4e99-9137-b5f15b7dcf31 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:13.178115 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3bd91b-bdab-4e99-9137-b5f15b7dcf31 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:13.179666 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 18/36] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 00:59:13.183981 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3bd91b-bdab-4e99-9137-b5f15b7dcf31 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:13.245180 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:13.253970 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:13.268892 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a75aab-d5dd-400a-b299-24d6958ec0b4 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:13.311782 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f1872-5d2d-48c3-91b1-46ae57900ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:13.312613 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f1872-5d2d-48c3-91b1-46ae57900ab2 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:13.312868 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f1872-5d2d-48c3-91b1-46ae57900ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:13.313141 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f1872-5d2d-48c3-91b1-46ae57900ab2 None admin] PATH_INFO: `/v3/users/51f97eb5e7894117a81b5bd04a11488e` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:13.314969 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3f1872-5d2d-48c3-91b1-46ae57900ab2 None admin] RBAC: Authorizing `identity:get_user(user_id=51f97eb5e7894117a81b5bd04a11488e)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:13.316514 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3f1872-5d2d-48c3-91b1-46ae57900ab2 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:13.317393 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3bd91b-bdab-4e99-9137-b5f15b7dcf31 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:13.320374 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 19/37] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 8 00:59:13 2022] GET /identity/v3/users/51f97eb5e7894117a81b5bd04a11488e => generated 247 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:13.320374 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c51178-0cf5-4b60-84a5-154d16d32946 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:13.320374 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c51178-0cf5-4b60-84a5-154d16d32946 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:13.320653 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c51178-0cf5-4b60-84a5-154d16d32946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:13.340754 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-55c51178-0cf5-4b60-84a5-154d16d32946 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:13.342574 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-55c51178-0cf5-4b60-84a5-154d16d32946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:13.358412 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac63476-3b0e-497c-a2a2-1e465829319e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:13.359123 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac63476-3b0e-497c-a2a2-1e465829319e None admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:13.359430 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac63476-3b0e-497c-a2a2-1e465829319e None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:13.359707 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac63476-3b0e-497c-a2a2-1e465829319e None admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:13.360407 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac63476-3b0e-497c-a2a2-1e465829319e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:13.362030 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac63476-3b0e-497c-a2a2-1e465829319e None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:13.398947 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 19/38] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Sep 8 00:59:13 2022] POST /identity/v3/projects => generated 301 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 00:59:13.400154 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52160c20-85a8-469d-8bc2-7985d4e82696 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:13.400448 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52160c20-85a8-469d-8bc2-7985d4e82696 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:13.400724 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52160c20-85a8-469d-8bc2-7985d4e82696 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:13.402060 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 20/39] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 00:59:13.402867 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52160c20-85a8-469d-8bc2-7985d4e82696 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:13.452714 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 20/40] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 00:59:13.454438 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725db8c-adf8-49c2-8e77-c31f437ed2f0 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:13.454917 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725db8c-adf8-49c2-8e77-c31f437ed2f0 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:13.455205 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725db8c-adf8-49c2-8e77-c31f437ed2f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:13.485413 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-b725db8c-adf8-49c2-8e77-c31f437ed2f0 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:13.487896 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-b725db8c-adf8-49c2-8e77-c31f437ed2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:13.547376 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:13.555051 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52160c20-85a8-469d-8bc2-7985d4e82696 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:13.592278 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 21/41] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 00:59:13.609983 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6109eb7-9960-4b92-b823-a0455a894d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:13.610714 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6109eb7-9960-4b92-b823-a0455a894d61 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:13.611071 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6109eb7-9960-4b92-b823-a0455a894d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:13.611422 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6109eb7-9960-4b92-b823-a0455a894d61 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:13.614326 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6109eb7-9960-4b92-b823-a0455a894d61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:13.614326 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6109eb7-9960-4b92-b823-a0455a894d61 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:13.615950 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 21/42] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 00:59:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:13.621540 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b725db8c-adf8-49c2-8e77-c31f437ed2f0 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:13.650340 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:13.734203 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6109eb7-9960-4b92-b823-a0455a894d61 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:13.738552 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b725db8c-adf8-49c2-8e77-c31f437ed2f0 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:13.775658 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d1ed3d-567c-4704-a2f7-32f788620541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:13.776362 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d1ed3d-567c-4704-a2f7-32f788620541 None admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:13.776645 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d1ed3d-567c-4704-a2f7-32f788620541 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:13.777001 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d1ed3d-567c-4704-a2f7-32f788620541 None admin] PATH_INFO: `/v3/domains/default/users/51f97eb5e7894117a81b5bd04a11488e/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:13.778948 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d1ed3d-567c-4704-a2f7-32f788620541 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=51f97eb5e7894117a81b5bd04a11488e, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:13.781153 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d1ed3d-567c-4704-a2f7-32f788620541 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:13.796590 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 22/43] 10.222.0.42 () {58 vars in 1361 bytes} [Thu Sep 8 00:59:13 2022] PUT /identity/v3/domains/default/users/51f97eb5e7894117a81b5bd04a11488e/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 00:59:13.847980 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6109eb7-9960-4b92-b823-a0455a894d61 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:13.887253 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fafd243-cb1c-4850-8899-86f996f021f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:13.887892 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fafd243-cb1c-4850-8899-86f996f021f8 None admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:13.887995 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d1ed3d-567c-4704-a2f7-32f788620541 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:13.888211 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fafd243-cb1c-4850-8899-86f996f021f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:13.888492 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fafd243-cb1c-4850-8899-86f996f021f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:13.889244 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fafd243-cb1c-4850-8899-86f996f021f8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:13.890827 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fafd243-cb1c-4850-8899-86f996f021f8 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:13.908582 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 22/44] 10.222.0.42 () {60 vars in 1131 bytes} [Thu Sep 8 00:59:13 2022] POST /identity/v3/roles => generated 223 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 00:59:13.988970 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:14.018040 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d1ed3d-567c-4704-a2f7-32f788620541 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:14.055609 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96455505-0bff-4acc-b2e2-ecad99a32223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:14.056287 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96455505-0bff-4acc-b2e2-ecad99a32223 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:14.056572 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96455505-0bff-4acc-b2e2-ecad99a32223 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:14.056896 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96455505-0bff-4acc-b2e2-ecad99a32223 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:14.058064 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96455505-0bff-4acc-b2e2-ecad99a32223 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:14.059563 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96455505-0bff-4acc-b2e2-ecad99a32223 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:14.061468 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 23/45] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 00:59:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 00:59:14.066984 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fafd243-cb1c-4850-8899-86f996f021f8 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:14.102731 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:14.285645 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fafd243-cb1c-4850-8899-86f996f021f8 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:14.320511 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ae44bc-d02b-4750-8c41-e2177864af67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:14.321121 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae44bc-d02b-4750-8c41-e2177864af67 None admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:14.321402 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae44bc-d02b-4750-8c41-e2177864af67 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:14.321674 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae44bc-d02b-4750-8c41-e2177864af67 None admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:14.322328 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ae44bc-d02b-4750-8c41-e2177864af67 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:14.323772 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ae44bc-d02b-4750-8c41-e2177864af67 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:14.349323 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 23/46] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Sep 8 00:59:14 2022] POST /identity/v3/projects => generated 312 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 00:59:15.971569 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe5064-8b59-4229-bc8b-f75a7709c24b None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:15.972088 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe5064-8b59-4229-bc8b-f75a7709c24b None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:15.972088 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe5064-8b59-4229-bc8b-f75a7709c24b None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:15.973429 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 24/47] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 00:59:15.978760 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c1e66-2469-4f29-a460-aa6e84e73fe1 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:15.979168 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c1e66-2469-4f29-a460-aa6e84e73fe1 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:15.979303 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c1e66-2469-4f29-a460-aa6e84e73fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:15.997350 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-032c1e66-2469-4f29-a460-aa6e84e73fe1 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:15.999582 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-032c1e66-2469-4f29-a460-aa6e84e73fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:16.065120 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 24/48] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 00:59:16.335368 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe5064-8b59-4229-bc8b-f75a7709c24b None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:16.440615 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe5064-8b59-4229-bc8b-f75a7709c24b None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:16.474326 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf9362-41f1-4b25-ab25-8aa529beebf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:16.474960 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf9362-41f1-4b25-ab25-8aa529beebf1 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:16.475244 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf9362-41f1-4b25-ab25-8aa529beebf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:16.475522 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf9362-41f1-4b25-ab25-8aa529beebf1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:16.487201 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaf9362-41f1-4b25-ab25-8aa529beebf1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 00:59:16.493317 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-8aaf9362-41f1-4b25-ab25-8aa529beebf1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 00:59:16.493317 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 00:59:16.493317 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 00:59:16.493317 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 00:59:16.493317 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 00:59:16.493317 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 00:59:16.493317 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 00:59:16.493317 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 00:59:16.493317 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 00:59:16.493317 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 00:59:16.493317 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 00:59:16.493317 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 00:59:16.493317 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 00:59:16.493317 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 00:59:16.493317 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 00:59:16.493317 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 00:59:16.494158 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 00:59:16.494158 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 00:59:16.494158 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 00:59:16.494158 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 00:59:16.494158 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 00:59:16.494158 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 00:59:16.494158 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 00:59:16.494158 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 00:59:16.494158 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 00:59:16.494158 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 00:59:16.494158 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 00:59:16.494158 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 00:59:16.494158 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 00:59:16.494158 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 00:59:16.494158 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 00:59:16.494158 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 00:59:16.494954 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 00:59:16.494954 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 00:59:16.494954 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 00:59:16.494954 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 00:59:16.494954 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 00:59:16.494954 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 00:59:16.494954 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 00:59:16.494954 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 00:59:16.494954 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 00:59:16.494954 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 00:59:16.494954 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 00:59:16.494954 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 00:59:16.494954 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 25/49] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 8 00:59:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 00:59:16.500238 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c1e66-2469-4f29-a460-aa6e84e73fe1 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:16.510326 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd832f9-f494-49e4-b708-05acdf03323a None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:16.510621 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd832f9-f494-49e4-b708-05acdf03323a None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:16.510777 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:16.510896 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd832f9-f494-49e4-b708-05acdf03323a None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:16.512289 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 26/50] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 00:59:16.518135 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b322f7f-6f64-4b7a-824f-06102d01b34b None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:16.518414 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b322f7f-6f64-4b7a-824f-06102d01b34b None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:16.518689 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b322f7f-6f64-4b7a-824f-06102d01b34b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:16.526055 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:16.538535 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-1b322f7f-6f64-4b7a-824f-06102d01b34b None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:16.539733 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-1b322f7f-6f64-4b7a-824f-06102d01b34b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:16.600100 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c1e66-2469-4f29-a460-aa6e84e73fe1 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:16.606571 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 27/51] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 00:59:16.636408 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c10994-d85d-401b-a199-f737ef80d53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:16.637026 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c10994-d85d-401b-a199-f737ef80d53c None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:16.637308 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c10994-d85d-401b-a199-f737ef80d53c None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:16.637584 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c10994-d85d-401b-a199-f737ef80d53c None admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:16.638254 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c10994-d85d-401b-a199-f737ef80d53c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:16.638561 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c10994-d85d-401b-a199-f737ef80d53c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:16.640267 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c10994-d85d-401b-a199-f737ef80d53c None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:16.648391 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 25/52] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 00:59:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 00:59:16.654004 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b322f7f-6f64-4b7a-824f-06102d01b34b None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:16.756424 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b322f7f-6f64-4b7a-824f-06102d01b34b None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:16.880569 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c10994-d85d-401b-a199-f737ef80d53c None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:16.896401 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7af287-19bf-4af2-8177-c2f45947bef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:16.897030 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7af287-19bf-4af2-8177-c2f45947bef5 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:16.897308 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7af287-19bf-4af2-8177-c2f45947bef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:16.897579 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7af287-19bf-4af2-8177-c2f45947bef5 None admin] PATH_INFO: `/v3/users/51f97eb5e7894117a81b5bd04a11488e` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:16.898686 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7af287-19bf-4af2-8177-c2f45947bef5 None admin] RBAC: Authorizing `identity:get_user(user_id=51f97eb5e7894117a81b5bd04a11488e)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:16.900151 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7af287-19bf-4af2-8177-c2f45947bef5 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:16.901716 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 28/53] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 8 00:59:16 2022] GET /identity/v3/users/51f97eb5e7894117a81b5bd04a11488e => generated 247 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:16.907905 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7af287-19bf-4af2-8177-c2f45947bef5 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:16.911056 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:16.950706 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:16.983473 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c10994-d85d-401b-a199-f737ef80d53c None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:17.009126 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7af287-19bf-4af2-8177-c2f45947bef5 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:17.016889 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0891259c-976d-4069-a91b-12f07ee1beeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:17.017178 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0891259c-976d-4069-a91b-12f07ee1beeb None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:17.017460 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0891259c-976d-4069-a91b-12f07ee1beeb None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:17.017732 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0891259c-976d-4069-a91b-12f07ee1beeb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:17.018847 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0891259c-976d-4069-a91b-12f07ee1beeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:17.020344 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0891259c-976d-4069-a91b-12f07ee1beeb None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:17.021912 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 26/54] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 00:59:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 00:59:17.027457 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0891259c-976d-4069-a91b-12f07ee1beeb None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:17.044025 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16747031-0a24-4534-9d86-53b20a470d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:17.044692 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16747031-0a24-4534-9d86-53b20a470d74 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:17.044975 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16747031-0a24-4534-9d86-53b20a470d74 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:17.045332 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16747031-0a24-4534-9d86-53b20a470d74 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:17.046617 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16747031-0a24-4534-9d86-53b20a470d74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:17.048068 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16747031-0a24-4534-9d86-53b20a470d74 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:17.049680 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 29/55] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 00:59:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 00:59:17.055309 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16747031-0a24-4534-9d86-53b20a470d74 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:17.125574 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:17.127556 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0891259c-976d-4069-a91b-12f07ee1beeb None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:17.159701 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16747031-0a24-4534-9d86-53b20a470d74 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:17.162535 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625b0080-e5b8-4318-969f-47bccf242b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:17.163147 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625b0080-e5b8-4318-969f-47bccf242b20 None admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:17.163432 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625b0080-e5b8-4318-969f-47bccf242b20 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:17.163708 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625b0080-e5b8-4318-969f-47bccf242b20 None admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:17.164380 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625b0080-e5b8-4318-969f-47bccf242b20 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:17.165855 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625b0080-e5b8-4318-969f-47bccf242b20 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:17.176342 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:17.193889 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8114805-2b08-4bc2-b98c-2cd1450b37a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:17.194328 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 27/56] 10.222.0.42 () {60 vars in 1140 bytes} [Thu Sep 8 00:59:17 2022] POST /identity/v3/projects => generated 298 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 00:59:17.194504 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8114805-2b08-4bc2-b98c-2cd1450b37a6 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:17.194789 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8114805-2b08-4bc2-b98c-2cd1450b37a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:17.195066 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8114805-2b08-4bc2-b98c-2cd1450b37a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:17.195802 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8114805-2b08-4bc2-b98c-2cd1450b37a6 None admin] RBAC: Adding query filter params (role.id=34de369ec7c54715aa83a9052a26c3f2, scope.domain.id=default, user.id=51f97eb5e7894117a81b5bd04a11488e) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:17.196115 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8114805-2b08-4bc2-b98c-2cd1450b37a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:17.197599 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8114805-2b08-4bc2-b98c-2cd1450b37a6 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:17.207446 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 30/57] 10.222.0.42 () {56 vars in 1322 bytes} [Thu Sep 8 00:59:17 2022] GET /identity/v3/role_assignments?user.id=51f97eb5e7894117a81b5bd04a11488e&scope.domain.id=default&role.id=34de369ec7c54715aa83a9052a26c3f2 => generated 530 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 00:59:19.365530 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10ab24-073a-44fa-a25c-0152dfde5e2b None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:19.366040 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10ab24-073a-44fa-a25c-0152dfde5e2b None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:19.366125 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10ab24-073a-44fa-a25c-0152dfde5e2b None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:19.367420 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 28/58] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 00:59:19.372684 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d536f548-3725-41b1-ac40-df1054a9623b None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:19.373132 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d536f548-3725-41b1-ac40-df1054a9623b None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:19.373836 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d536f548-3725-41b1-ac40-df1054a9623b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:19.391688 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-d536f548-3725-41b1-ac40-df1054a9623b None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:19.392910 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-d536f548-3725-41b1-ac40-df1054a9623b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:19.456624 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 31/59] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 00:59:19.725244 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef10ab24-073a-44fa-a25c-0152dfde5e2b None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:19.830914 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef10ab24-073a-44fa-a25c-0152dfde5e2b None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:19.864495 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81305fe6-76e6-4fb2-9c05-b83f332700ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:19.865121 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81305fe6-76e6-4fb2-9c05-b83f332700ef None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:19.865408 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81305fe6-76e6-4fb2-9c05-b83f332700ef None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:19.865685 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81305fe6-76e6-4fb2-9c05-b83f332700ef None admin] PATH_INFO: `/v3/domains/default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:19.866802 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81305fe6-76e6-4fb2-9c05-b83f332700ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:19.868308 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81305fe6-76e6-4fb2-9c05-b83f332700ef None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:19.869865 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 29/60] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 00:59:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:19.875542 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d536f548-3725-41b1-ac40-df1054a9623b None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:19.935494 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:19.972229 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d536f548-3725-41b1-ac40-df1054a9623b None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:20.005145 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7684011-5a6c-432e-801a-1a7df242759a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:20.005749 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7684011-5a6c-432e-801a-1a7df242759a None admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:20.006032 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7684011-5a6c-432e-801a-1a7df242759a None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:20.006308 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7684011-5a6c-432e-801a-1a7df242759a None admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:20.006967 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7684011-5a6c-432e-801a-1a7df242759a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:20.008454 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7684011-5a6c-432e-801a-1a7df242759a None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:20.035795 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 32/61] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 8 00:59:19 2022] POST /identity/v3/users => generated 275 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 00:59:23.243230 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8efea4-5124-48eb-a401-91f3977d5da9 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:23.244241 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8efea4-5124-48eb-a401-91f3977d5da9 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:23.244743 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8efea4-5124-48eb-a401-91f3977d5da9 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:23.246588 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 30/62] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 00:59:23.252516 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a5294b-5a60-4fd6-96dc-ab1097d8b29d None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:23.253385 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a5294b-5a60-4fd6-96dc-ab1097d8b29d None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:23.253851 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a5294b-5a60-4fd6-96dc-ab1097d8b29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:23.273140 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-50a5294b-5a60-4fd6-96dc-ab1097d8b29d None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:23.274701 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-50a5294b-5a60-4fd6-96dc-ab1097d8b29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:23.279805 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eb8801-c3db-41e0-ac85-fa94d4a88cad None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:23.280445 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eb8801-c3db-41e0-ac85-fa94d4a88cad None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:23.280915 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eb8801-c3db-41e0-ac85-fa94d4a88cad None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:23.282429 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 31/63] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 00:59:23.287333 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b8a8d-f5d5-4551-94ec-5f0dccb3d624 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:23.287822 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b8a8d-f5d5-4551-94ec-5f0dccb3d624 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:23.288314 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b8a8d-f5d5-4551-94ec-5f0dccb3d624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:23.308415 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-b14b8a8d-f5d5-4551-94ec-5f0dccb3d624 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:23.309941 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-b14b8a8d-f5d5-4551-94ec-5f0dccb3d624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:23.380493 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:23.408199 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 32/64] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 00:59:23.409458 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 33/65] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 00:59:23.712400 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a51917-079a-436b-a83f-ef781b676c8d None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:23.713061 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a51917-079a-436b-a83f-ef781b676c8d None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:23.713554 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a51917-079a-436b-a83f-ef781b676c8d None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:23.715314 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 33/66] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 00:59:23.721900 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1a9b31-ee4f-4624-a3d4-b59240981f84 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:23.722501 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1a9b31-ee4f-4624-a3d4-b59240981f84 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:23.722967 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1a9b31-ee4f-4624-a3d4-b59240981f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:23.727217 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a51917-079a-436b-a83f-ef781b676c8d None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:23.758761 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-bf1a9b31-ee4f-4624-a3d4-b59240981f84 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:23.760638 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-bf1a9b31-ee4f-4624-a3d4-b59240981f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:23.816189 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:23.839824 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 34/67] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 00:59:23.841022 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1a9b31-ee4f-4624-a3d4-b59240981f84 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:23.891772 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a51917-079a-436b-a83f-ef781b676c8d None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:23.939756 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a938e8f3-bf47-4359-b29c-44abcb66ae94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:23.940449 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938e8f3-bf47-4359-b29c-44abcb66ae94 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:23.940984 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938e8f3-bf47-4359-b29c-44abcb66ae94 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:23.941279 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938e8f3-bf47-4359-b29c-44abcb66ae94 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:23.957060 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a938e8f3-bf47-4359-b29c-44abcb66ae94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 00:59:23.961938 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1a9b31-ee4f-4624-a3d4-b59240981f84 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:23.963137 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-a938e8f3-bf47-4359-b29c-44abcb66ae94 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 00:59:23.963137 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 00:59:23.963137 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 00:59:23.963137 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 00:59:23.963137 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 00:59:23.963137 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 00:59:23.963137 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 00:59:23.963137 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 00:59:23.963137 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 00:59:23.963137 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 00:59:23.963137 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 00:59:23.963137 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 00:59:23.963137 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 00:59:23.963137 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 00:59:23.963137 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 00:59:23.963137 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 00:59:23.963968 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 00:59:23.963968 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 00:59:23.963968 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 00:59:23.963968 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 00:59:23.963968 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 00:59:23.963968 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 00:59:23.963968 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 00:59:23.963968 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 00:59:23.963968 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 00:59:23.963968 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 00:59:23.963968 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 00:59:23.963968 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 00:59:23.963968 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 00:59:23.963968 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 00:59:23.963968 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 00:59:23.963968 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 00:59:23.964758 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 00:59:23.964758 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 00:59:23.964758 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 00:59:23.964758 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 00:59:23.964758 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 00:59:23.964758 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 00:59:23.964758 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 00:59:23.964758 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 00:59:23.964758 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 00:59:23.964758 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 00:59:23.964758 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 00:59:23.964758 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 00:59:23.964758 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 34/68] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 8 00:59:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 00:59:23.966047 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd818ae-2b51-4cc4-903a-a9acdf5584d3 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:23.966625 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd818ae-2b51-4cc4-903a-a9acdf5584d3 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:23.968214 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd818ae-2b51-4cc4-903a-a9acdf5584d3 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:23.970679 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 35/69] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 00:59:23.976169 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd818ae-2b51-4cc4-903a-a9acdf5584d3 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:24.003410 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce29ea3-113a-4de9-a2aa-0af6d2ef6b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:24.004213 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce29ea3-113a-4de9-a2aa-0af6d2ef6b6f None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:24.004596 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce29ea3-113a-4de9-a2aa-0af6d2ef6b6f None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:24.004973 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce29ea3-113a-4de9-a2aa-0af6d2ef6b6f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:24.017321 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce29ea3-113a-4de9-a2aa-0af6d2ef6b6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 00:59:24.022494 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-0ce29ea3-113a-4de9-a2aa-0af6d2ef6b6f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 00:59:24.022494 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 00:59:24.022494 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 00:59:24.022494 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 00:59:24.022494 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 00:59:24.022494 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 00:59:24.022494 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 00:59:24.022494 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 00:59:24.022494 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 00:59:24.022494 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 00:59:24.022494 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 00:59:24.022494 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 00:59:24.022494 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 00:59:24.022494 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 00:59:24.022494 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 00:59:24.022494 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 00:59:24.023540 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 00:59:24.023540 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 00:59:24.023540 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 00:59:24.023540 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 00:59:24.023540 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 00:59:24.023540 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 00:59:24.023540 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 00:59:24.023540 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 00:59:24.023540 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 00:59:24.023540 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 00:59:24.023540 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 00:59:24.023540 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 00:59:24.023540 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 00:59:24.023540 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 00:59:24.023540 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 00:59:24.023540 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 00:59:24.024525 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 00:59:24.024525 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 00:59:24.024525 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 00:59:24.024525 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 00:59:24.024525 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 00:59:24.024525 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 00:59:24.024525 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 00:59:24.024525 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 00:59:24.024525 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 00:59:24.024525 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 00:59:24.024525 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 00:59:24.024525 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 00:59:24.027058 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 35/70] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 00:59:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 00:59:24.028292 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af4b5f-dfcb-457b-963a-b2a7d41bf392 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:24.028673 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af4b5f-dfcb-457b-963a-b2a7d41bf392 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:24.029044 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af4b5f-dfcb-457b-963a-b2a7d41bf392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:24.044502 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:24.051007 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-b0af4b5f-dfcb-457b-963a-b2a7d41bf392 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:24.053040 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-b0af4b5f-dfcb-457b-963a-b2a7d41bf392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:24.093530 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:24.117786 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd818ae-2b51-4cc4-903a-a9acdf5584d3 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:24.131579 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 36/71] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 00:59:24.136450 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af4b5f-dfcb-457b-963a-b2a7d41bf392 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:24.169125 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91769232-98e9-4acc-83d8-a3312c31e305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:24.170043 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91769232-98e9-4acc-83d8-a3312c31e305 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:24.170423 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91769232-98e9-4acc-83d8-a3312c31e305 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:24.170699 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91769232-98e9-4acc-83d8-a3312c31e305 None admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:24.171369 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91769232-98e9-4acc-83d8-a3312c31e305 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:24.171902 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91769232-98e9-4acc-83d8-a3312c31e305 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:24.174213 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91769232-98e9-4acc-83d8-a3312c31e305 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:24.184402 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 36/72] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 00:59:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:24.186988 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91769232-98e9-4acc-83d8-a3312c31e305 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:24.247100 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af4b5f-dfcb-457b-963a-b2a7d41bf392 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:24.283917 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170da545-d83d-47d6-babb-eb0f6f3409cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:24.285275 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170da545-d83d-47d6-babb-eb0f6f3409cd None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:24.285700 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170da545-d83d-47d6-babb-eb0f6f3409cd None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:24.286168 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170da545-d83d-47d6-babb-eb0f6f3409cd None admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:24.286973 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170da545-d83d-47d6-babb-eb0f6f3409cd None admin] RBAC: Adding query filter params (name=member) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:24.287478 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170da545-d83d-47d6-babb-eb0f6f3409cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:24.289050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170da545-d83d-47d6-babb-eb0f6f3409cd None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:24.298898 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 37/73] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 8 00:59:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:24.300071 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170da545-d83d-47d6-babb-eb0f6f3409cd None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:24.338180 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91769232-98e9-4acc-83d8-a3312c31e305 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:24.343782 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:24.350015 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:24.375983 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5403852-18bc-467a-b3d4-78b32e0435fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:24.376646 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5403852-18bc-467a-b3d4-78b32e0435fa None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:24.376929 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5403852-18bc-467a-b3d4-78b32e0435fa None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:24.377199 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5403852-18bc-467a-b3d4-78b32e0435fa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:24.387511 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5403852-18bc-467a-b3d4-78b32e0435fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 00:59:24.392953 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-b5403852-18bc-467a-b3d4-78b32e0435fa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 00:59:24.392953 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 00:59:24.392953 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 00:59:24.392953 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 00:59:24.392953 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 00:59:24.392953 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 00:59:24.392953 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 00:59:24.392953 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 00:59:24.392953 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 00:59:24.392953 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 00:59:24.392953 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 00:59:24.392953 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 00:59:24.392953 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 00:59:24.392953 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 00:59:24.392953 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 00:59:24.392953 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 00:59:24.393787 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 00:59:24.393787 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 00:59:24.393787 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 00:59:24.393787 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 00:59:24.393787 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 00:59:24.393787 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 00:59:24.393787 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 00:59:24.393787 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 00:59:24.393787 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 00:59:24.393787 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 00:59:24.393787 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 00:59:24.393787 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 00:59:24.393787 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 00:59:24.393787 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 00:59:24.393787 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 00:59:24.393787 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 00:59:24.394547 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 00:59:24.394547 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 00:59:24.394547 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 00:59:24.394547 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 00:59:24.394547 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 00:59:24.394547 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 00:59:24.394547 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 00:59:24.394547 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 00:59:24.394547 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 00:59:24.394547 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 00:59:24.394547 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 00:59:24.394547 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 00:59:24.399077 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 37/74] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 8 00:59:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 00:59:24.399773 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5403852-18bc-467a-b3d4-78b32e0435fa None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:24.420338 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170da545-d83d-47d6-babb-eb0f6f3409cd None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:24.455907 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d9206c-dbb7-47f3-bdd9-cb6a783b970c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:24.456570 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d9206c-dbb7-47f3-bdd9-cb6a783b970c None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:24.456856 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d9206c-dbb7-47f3-bdd9-cb6a783b970c None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:24.457132 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d9206c-dbb7-47f3-bdd9-cb6a783b970c None admin] PATH_INFO: `/v3/users/51f97eb5e7894117a81b5bd04a11488e` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:24.458273 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d9206c-dbb7-47f3-bdd9-cb6a783b970c None admin] RBAC: Authorizing `identity:get_user(user_id=51f97eb5e7894117a81b5bd04a11488e)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:24.459751 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d9206c-dbb7-47f3-bdd9-cb6a783b970c None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:24.461389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 38/75] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 8 00:59:24 2022] GET /identity/v3/users/51f97eb5e7894117a81b5bd04a11488e => generated 247 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:24.462498 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be49b1dd-cd0f-4831-aad2-b756df65c0df None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:24.462781 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be49b1dd-cd0f-4831-aad2-b756df65c0df None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:24.463064 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be49b1dd-cd0f-4831-aad2-b756df65c0df None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:24.464283 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 39/76] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 00:59:24.464981 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be49b1dd-cd0f-4831-aad2-b756df65c0df None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:24.509472 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5403852-18bc-467a-b3d4-78b32e0435fa None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:24.542549 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7ee499-cc1a-4923-8dc8-40f6f84d8029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:24.543177 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ee499-cc1a-4923-8dc8-40f6f84d8029 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:24.543467 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ee499-cc1a-4923-8dc8-40f6f84d8029 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:24.543742 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ee499-cc1a-4923-8dc8-40f6f84d8029 None admin] PATH_INFO: `/v3/users/3c115932304948d4968c4f07fae9c96b` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:24.557190 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7ee499-cc1a-4923-8dc8-40f6f84d8029 None admin] RBAC: Authorizing `identity:get_user(user_id=3c115932304948d4968c4f07fae9c96b)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:24.558647 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7ee499-cc1a-4923-8dc8-40f6f84d8029 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:24.560277 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 38/77] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 8 00:59:24 2022] GET /identity/v3/users/3c115932304948d4968c4f07fae9c96b => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:24.561608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7ee499-cc1a-4923-8dc8-40f6f84d8029 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:24.568884 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:24.573127 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be49b1dd-cd0f-4831-aad2-b756df65c0df None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:24.588419 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:24.608948 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490ceddb-cb1e-49f0-83b9-f0e828f8e2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:24.609551 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ceddb-cb1e-49f0-83b9-f0e828f8e2c1 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:24.609833 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ceddb-cb1e-49f0-83b9-f0e828f8e2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:24.610105 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ceddb-cb1e-49f0-83b9-f0e828f8e2c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:24.610766 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490ceddb-cb1e-49f0-83b9-f0e828f8e2c1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:24.611064 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490ceddb-cb1e-49f0-83b9-f0e828f8e2c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:24.612477 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490ceddb-cb1e-49f0-83b9-f0e828f8e2c1 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:24.620290 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 40/78] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 00:59:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:24.621086 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490ceddb-cb1e-49f0-83b9-f0e828f8e2c1 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:24.665504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7ee499-cc1a-4923-8dc8-40f6f84d8029 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:24.700331 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ac5164-9401-413b-811e-273c0fdc5605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:24.700891 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac5164-9401-413b-811e-273c0fdc5605 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:24.701203 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac5164-9401-413b-811e-273c0fdc5605 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:24.701474 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac5164-9401-413b-811e-273c0fdc5605 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:24.711547 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ac5164-9401-413b-811e-273c0fdc5605 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 00:59:24.716405 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-08ac5164-9401-413b-811e-273c0fdc5605 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 00:59:24.716405 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 00:59:24.716405 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 00:59:24.716405 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 00:59:24.716405 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 00:59:24.716405 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 00:59:24.716405 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 00:59:24.716405 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 00:59:24.716405 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 00:59:24.716405 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 00:59:24.716405 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 00:59:24.716405 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 00:59:24.716405 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 00:59:24.716405 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 00:59:24.716405 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 00:59:24.716405 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 00:59:24.717408 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 00:59:24.717408 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 00:59:24.717408 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 00:59:24.717408 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 00:59:24.717408 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 00:59:24.717408 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 00:59:24.717408 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 00:59:24.717408 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 00:59:24.717408 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 00:59:24.717408 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 00:59:24.717408 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 00:59:24.717408 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 00:59:24.717408 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 00:59:24.717408 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 00:59:24.717408 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 00:59:24.717408 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 00:59:24.718175 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 00:59:24.718175 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 00:59:24.718175 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 00:59:24.718175 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 00:59:24.718175 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 00:59:24.718175 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 00:59:24.718175 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 00:59:24.718175 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 00:59:24.718175 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 00:59:24.718175 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 00:59:24.718175 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 00:59:24.718175 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 00:59:24.718175 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 39/79] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 00:59:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 00:59:24.718792 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5284c099-c832-447b-9688-7849b87f3566 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:24.719108 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5284c099-c832-447b-9688-7849b87f3566 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:24.719194 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5284c099-c832-447b-9688-7849b87f3566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:24.722848 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490ceddb-cb1e-49f0-83b9-f0e828f8e2c1 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:24.736824 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-5284c099-c832-447b-9688-7849b87f3566 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:24.738024 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-5284c099-c832-447b-9688-7849b87f3566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:24.757260 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f68933-5455-43f1-983f-a7f1c9be61ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:24.757876 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f68933-5455-43f1-983f-a7f1c9be61ff None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:24.758157 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f68933-5455-43f1-983f-a7f1c9be61ff None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:24.758432 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f68933-5455-43f1-983f-a7f1c9be61ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:24.787611 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:24.805537 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 40/80] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 00:59:24.806381 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5284c099-c832-447b-9688-7849b87f3566 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:24.828055 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:24.837564 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f68933-5455-43f1-983f-a7f1c9be61ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:24.839082 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f68933-5455-43f1-983f-a7f1c9be61ff None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:24.910763 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5284c099-c832-447b-9688-7849b87f3566 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:24.941171 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 41/81] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 00:59:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 00:59:24.943846 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a288ca3-96f9-4532-a2ea-5619aa4a6bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:24.944487 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a288ca3-96f9-4532-a2ea-5619aa4a6bcc None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:24.944769 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a288ca3-96f9-4532-a2ea-5619aa4a6bcc None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:24.945053 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a288ca3-96f9-4532-a2ea-5619aa4a6bcc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:24.946338 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f68933-5455-43f1-983f-a7f1c9be61ff None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:25.010776 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:25.020184 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a288ca3-96f9-4532-a2ea-5619aa4a6bcc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:25.021779 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a288ca3-96f9-4532-a2ea-5619aa4a6bcc None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:25.041786 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:25.051049 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f68933-5455-43f1-983f-a7f1c9be61ff None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:25.085695 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091ecc2c-80dc-4171-be1d-5195ccd137ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:25.086322 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ecc2c-80dc-4171-be1d-5195ccd137ac None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:25.086626 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ecc2c-80dc-4171-be1d-5195ccd137ac None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:25.086897 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ecc2c-80dc-4171-be1d-5195ccd137ac None admin] PATH_INFO: `/v3/users/3c115932304948d4968c4f07fae9c96b` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:25.088018 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091ecc2c-80dc-4171-be1d-5195ccd137ac None admin] RBAC: Authorizing `identity:get_user(user_id=3c115932304948d4968c4f07fae9c96b)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:25.089511 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091ecc2c-80dc-4171-be1d-5195ccd137ac None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:25.091086 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 42/82] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 8 00:59:24 2022] GET /identity/v3/users/3c115932304948d4968c4f07fae9c96b => generated 275 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:25.092387 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091ecc2c-80dc-4171-be1d-5195ccd137ac None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:25.125492 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 41/83] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 00:59:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 00:59:25.129218 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a288ca3-96f9-4532-a2ea-5619aa4a6bcc None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:25.194887 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091ecc2c-80dc-4171-be1d-5195ccd137ac None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:25.223365 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:25.229696 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42e1241-eff1-4f1a-9e51-aa6a4f6646ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:25.229873 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a288ca3-96f9-4532-a2ea-5619aa4a6bcc None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:25.230323 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e1241-eff1-4f1a-9e51-aa6a4f6646ca None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:25.230601 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e1241-eff1-4f1a-9e51-aa6a4f6646ca None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:25.230869 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e1241-eff1-4f1a-9e51-aa6a4f6646ca None admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:25.231577 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42e1241-eff1-4f1a-9e51-aa6a4f6646ca None admin] RBAC: Adding query filter params (name=member) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:25.231886 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42e1241-eff1-4f1a-9e51-aa6a4f6646ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:25.233435 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42e1241-eff1-4f1a-9e51-aa6a4f6646ca None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:25.241344 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 43/84] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 8 00:59:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:25.242121 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42e1241-eff1-4f1a-9e51-aa6a4f6646ca None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:25.250999 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:25.268565 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2868218-8aca-429e-9bc6-24046bd8265a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:25.269133 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2868218-8aca-429e-9bc6-24046bd8265a None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:25.269416 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2868218-8aca-429e-9bc6-24046bd8265a None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:25.269689 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2868218-8aca-429e-9bc6-24046bd8265a None admin] PATH_INFO: `/v3/projects/41f15a3cf54f4397920b0e8f758539ea` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:25.270826 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2868218-8aca-429e-9bc6-24046bd8265a None admin] RBAC: Authorizing `identity:get_project(project_id=41f15a3cf54f4397920b0e8f758539ea)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:25.272320 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2868218-8aca-429e-9bc6-24046bd8265a None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:25.273948 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 42/85] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 00:59:25 2022] GET /identity/v3/projects/41f15a3cf54f4397920b0e8f758539ea => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:25.274704 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2868218-8aca-429e-9bc6-24046bd8265a None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:25.345863 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42e1241-eff1-4f1a-9e51-aa6a4f6646ca None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:25.373472 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2868218-8aca-429e-9bc6-24046bd8265a None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:25.379488 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82d63cd-9050-405d-bde8-ebf8973a53dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:25.380120 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d63cd-9050-405d-bde8-ebf8973a53dd None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:25.380433 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d63cd-9050-405d-bde8-ebf8973a53dd None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:25.380712 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d63cd-9050-405d-bde8-ebf8973a53dd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:25.381816 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82d63cd-9050-405d-bde8-ebf8973a53dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:25.383270 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82d63cd-9050-405d-bde8-ebf8973a53dd None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:25.384865 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 44/86] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 00:59:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:25.385882 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82d63cd-9050-405d-bde8-ebf8973a53dd None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:25.409955 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c627d86c-e0bf-4447-8dfd-cd4586eecb58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:25.410559 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c627d86c-e0bf-4447-8dfd-cd4586eecb58 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:25.410838 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c627d86c-e0bf-4447-8dfd-cd4586eecb58 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:25.411111 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c627d86c-e0bf-4447-8dfd-cd4586eecb58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:25.445578 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:25.465890 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:25.479079 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c627d86c-e0bf-4447-8dfd-cd4586eecb58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:25.480589 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c627d86c-e0bf-4447-8dfd-cd4586eecb58 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:25.486383 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82d63cd-9050-405d-bde8-ebf8973a53dd None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:25.519637 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af232619-378a-4d3a-9c9a-0c120d84462b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:25.520223 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af232619-378a-4d3a-9c9a-0c120d84462b None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:25.520510 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af232619-378a-4d3a-9c9a-0c120d84462b None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:25.520785 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af232619-378a-4d3a-9c9a-0c120d84462b None admin] PATH_INFO: `/v3/projects/41f15a3cf54f4397920b0e8f758539ea` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:25.521888 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af232619-378a-4d3a-9c9a-0c120d84462b None admin] RBAC: Authorizing `identity:get_project(project_id=41f15a3cf54f4397920b0e8f758539ea)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:25.523317 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af232619-378a-4d3a-9c9a-0c120d84462b None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:25.524965 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 45/87] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 00:59:25 2022] GET /identity/v3/projects/41f15a3cf54f4397920b0e8f758539ea => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:25.525715 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af232619-378a-4d3a-9c9a-0c120d84462b None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:25.580594 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 43/88] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 00:59:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 00:59:25.581479 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c627d86c-e0bf-4447-8dfd-cd4586eecb58 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:25.626085 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af232619-378a-4d3a-9c9a-0c120d84462b None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:25.654980 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:25.663293 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7349eb6-9d79-4d80-a564-51443581b402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:25.663924 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7349eb6-9d79-4d80-a564-51443581b402 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:25.664236 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7349eb6-9d79-4d80-a564-51443581b402 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:25.664510 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7349eb6-9d79-4d80-a564-51443581b402 None admin] PATH_INFO: `/v3/users/3c115932304948d4968c4f07fae9c96b` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:25.665222 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:25.665634 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7349eb6-9d79-4d80-a564-51443581b402 None admin] RBAC: Authorizing `identity:get_user(user_id=3c115932304948d4968c4f07fae9c96b)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:25.667187 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7349eb6-9d79-4d80-a564-51443581b402 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:25.673374 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 46/89] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 8 00:59:25 2022] GET /identity/v3/users/3c115932304948d4968c4f07fae9c96b => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:25.673374 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7349eb6-9d79-4d80-a564-51443581b402 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:25.680569 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c627d86c-e0bf-4447-8dfd-cd4586eecb58 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:25.714369 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1114fde3-f4ab-4812-8661-006f726e9205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:25.714978 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114fde3-f4ab-4812-8661-006f726e9205 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:25.715264 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114fde3-f4ab-4812-8661-006f726e9205 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:25.715539 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114fde3-f4ab-4812-8661-006f726e9205 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:25.716300 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1114fde3-f4ab-4812-8661-006f726e9205 None admin] RBAC: Adding query filter params (role.id=34de369ec7c54715aa83a9052a26c3f2, scope.project.id=41f15a3cf54f4397920b0e8f758539ea, user.id=51f97eb5e7894117a81b5bd04a11488e) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:25.716616 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1114fde3-f4ab-4812-8661-006f726e9205 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:25.718731 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1114fde3-f4ab-4812-8661-006f726e9205 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:25.724064 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 44/90] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 8 00:59:25 2022] GET /identity/v3/role_assignments?user.id=51f97eb5e7894117a81b5bd04a11488e&scope.project.id=41f15a3cf54f4397920b0e8f758539ea&role.id=34de369ec7c54715aa83a9052a26c3f2 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:25.724876 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1114fde3-f4ab-4812-8661-006f726e9205 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:25.780557 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7349eb6-9d79-4d80-a564-51443581b402 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:25.817885 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9bc9c3-00c6-4e8b-b183-089d220085eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:25.818666 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9bc9c3-00c6-4e8b-b183-089d220085eb None admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:25.819020 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9bc9c3-00c6-4e8b-b183-089d220085eb None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:25.819365 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9bc9c3-00c6-4e8b-b183-089d220085eb None admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:25.820051 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9bc9c3-00c6-4e8b-b183-089d220085eb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:25.821580 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9bc9c3-00c6-4e8b-b183-089d220085eb None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:25.839491 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1114fde3-f4ab-4812-8661-006f726e9205 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:25.856181 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 47/91] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 8 00:59:25 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 00:59:25.856181 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9bc9c3-00c6-4e8b-b183-089d220085eb None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:25.914436 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c0eed7-8917-4b1f-a1fd-6d4cfc135a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:25.915082 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0eed7-8917-4b1f-a1fd-6d4cfc135a97 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:25.915368 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0eed7-8917-4b1f-a1fd-6d4cfc135a97 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:25.915646 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0eed7-8917-4b1f-a1fd-6d4cfc135a97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:25.916577 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c0eed7-8917-4b1f-a1fd-6d4cfc135a97 None admin] RBAC: Adding query filter params (role.id=92aa6d6824404d10b4a3be8b4a6bc317, scope.project.id=41f15a3cf54f4397920b0e8f758539ea, user.id=3c115932304948d4968c4f07fae9c96b) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:25.916751 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c0eed7-8917-4b1f-a1fd-6d4cfc135a97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:25.918301 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c0eed7-8917-4b1f-a1fd-6d4cfc135a97 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:25.927954 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:25.940190 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 45/92] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 8 00:59:25 2022] GET /identity/v3/role_assignments?user.id=3c115932304948d4968c4f07fae9c96b&scope.project.id=41f15a3cf54f4397920b0e8f758539ea&role.id=92aa6d6824404d10b4a3be8b4a6bc317 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:25.941618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c0eed7-8917-4b1f-a1fd-6d4cfc135a97 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:25.994995 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9bc9c3-00c6-4e8b-b183-089d220085eb None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:26.000601 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:26.042554 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600085dc-4e45-409f-94f8-3f7d974454cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:26.042554 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600085dc-4e45-409f-94f8-3f7d974454cb None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:26.042554 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600085dc-4e45-409f-94f8-3f7d974454cb None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:26.042554 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600085dc-4e45-409f-94f8-3f7d974454cb None admin] PATH_INFO: `/v3/projects/41f15a3cf54f4397920b0e8f758539ea` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:26.043855 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600085dc-4e45-409f-94f8-3f7d974454cb None admin] RBAC: Authorizing `identity:get_project(project_id=41f15a3cf54f4397920b0e8f758539ea)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:26.046044 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600085dc-4e45-409f-94f8-3f7d974454cb None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:26.052176 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 48/93] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 00:59:25 2022] GET /identity/v3/projects/41f15a3cf54f4397920b0e8f758539ea => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:26.054043 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600085dc-4e45-409f-94f8-3f7d974454cb None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:26.140333 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c0eed7-8917-4b1f-a1fd-6d4cfc135a97 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:26.175979 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600085dc-4e45-409f-94f8-3f7d974454cb None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:26.182335 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b67552f-6f9f-4553-bd67-b5007d48ac02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:26.182335 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67552f-6f9f-4553-bd67-b5007d48ac02 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:26.182335 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67552f-6f9f-4553-bd67-b5007d48ac02 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:26.182609 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67552f-6f9f-4553-bd67-b5007d48ac02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:26.209509 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:26.228495 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac7655-2405-45b7-88b5-faf097134dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:26.229339 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac7655-2405-45b7-88b5-faf097134dc8 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:26.229840 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac7655-2405-45b7-88b5-faf097134dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:26.230127 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac7655-2405-45b7-88b5-faf097134dc8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:26.231856 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ac7655-2405-45b7-88b5-faf097134dc8 None admin] RBAC: Adding query filter params (role.id=069fe3e033664493b3def844e3735bc9, scope.project.id=41f15a3cf54f4397920b0e8f758539ea, user.id=3c115932304948d4968c4f07fae9c96b) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:26.234174 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ac7655-2405-45b7-88b5-faf097134dc8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:26.235712 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ac7655-2405-45b7-88b5-faf097134dc8 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:26.243260 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 49/94] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 8 00:59:26 2022] GET /identity/v3/role_assignments?user.id=3c115932304948d4968c4f07fae9c96b&scope.project.id=41f15a3cf54f4397920b0e8f758539ea&role.id=069fe3e033664493b3def844e3735bc9 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 00:59:26.276554 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b67552f-6f9f-4553-bd67-b5007d48ac02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:26.278248 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b67552f-6f9f-4553-bd67-b5007d48ac02 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:26.300612 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:26.395294 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 46/95] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 00:59:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 00:59:26.401265 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac7655-2405-45b7-88b5-faf097134dc8 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:26.523570 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac7655-2405-45b7-88b5-faf097134dc8 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:26.566442 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f892a60d-42c2-4e86-ac83-577146b7291b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:26.567025 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f892a60d-42c2-4e86-ac83-577146b7291b None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:26.567308 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f892a60d-42c2-4e86-ac83-577146b7291b None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:26.567588 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f892a60d-42c2-4e86-ac83-577146b7291b None admin] PATH_INFO: `/v3/projects/34348313aa84446d8c5844c3ed29655b` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:26.568772 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f892a60d-42c2-4e86-ac83-577146b7291b None admin] RBAC: Authorizing `identity:get_project(project_id=34348313aa84446d8c5844c3ed29655b)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:26.570256 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f892a60d-42c2-4e86-ac83-577146b7291b None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:26.572305 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 50/96] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 00:59:26 2022] GET /identity/v3/projects/34348313aa84446d8c5844c3ed29655b => generated 312 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:26.578377 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b67552f-6f9f-4553-bd67-b5007d48ac02 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:26.709195 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b67552f-6f9f-4553-bd67-b5007d48ac02 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:26.724406 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:26.752023 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6813b493-6266-4b73-9eab-569e6c8ba491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:26.752667 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813b493-6266-4b73-9eab-569e6c8ba491 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:26.752959 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813b493-6266-4b73-9eab-569e6c8ba491 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:26.753232 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813b493-6266-4b73-9eab-569e6c8ba491 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:26.753985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6813b493-6266-4b73-9eab-569e6c8ba491 None admin] RBAC: Adding query filter params (role.id=92aa6d6824404d10b4a3be8b4a6bc317, scope.project.id=34348313aa84446d8c5844c3ed29655b, user.id=3c115932304948d4968c4f07fae9c96b) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:26.754279 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6813b493-6266-4b73-9eab-569e6c8ba491 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:26.755731 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6813b493-6266-4b73-9eab-569e6c8ba491 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:26.762411 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 47/97] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 8 00:59:26 2022] GET /identity/v3/role_assignments?user.id=3c115932304948d4968c4f07fae9c96b&scope.project.id=34348313aa84446d8c5844c3ed29655b&role.id=92aa6d6824404d10b4a3be8b4a6bc317 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:28.460662 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216bae5c-f085-4517-a74f-7e80311242e6 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:28.461212 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216bae5c-f085-4517-a74f-7e80311242e6 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:28.461212 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216bae5c-f085-4517-a74f-7e80311242e6 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:28.462546 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 51/98] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 00:59:28.468152 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1c8c0e-c29a-4de8-a286-3de79784205b None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:28.468554 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1c8c0e-c29a-4de8-a286-3de79784205b None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:28.468781 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1c8c0e-c29a-4de8-a286-3de79784205b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:28.497143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-6b1c8c0e-c29a-4de8-a286-3de79784205b None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:28.498546 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-6b1c8c0e-c29a-4de8-a286-3de79784205b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:28.603662 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 48/99] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 00:59:28.844042 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c3467-8635-43dd-8030-d99920010913 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:28.844042 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c3467-8635-43dd-8030-d99920010913 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:28.844042 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c3467-8635-43dd-8030-d99920010913 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:28.844042 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 52/100] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 00:59:28.849318 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16de549-6fdf-4ec1-afed-a72056c8a95b None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:28.849615 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16de549-6fdf-4ec1-afed-a72056c8a95b None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:28.849899 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16de549-6fdf-4ec1-afed-a72056c8a95b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:28.873594 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-b16de549-6fdf-4ec1-afed-a72056c8a95b None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:28.874861 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-b16de549-6fdf-4ec1-afed-a72056c8a95b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:28.959417 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 49/101] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 00:59:29.155896 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432c3467-8635-43dd-8030-d99920010913 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:29.227518 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:29.287473 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432c3467-8635-43dd-8030-d99920010913 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:29.324291 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d646e2-50f4-47b7-aeb9-b9ee6fc7d49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:29.324931 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d646e2-50f4-47b7-aeb9-b9ee6fc7d49b None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:29.325214 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d646e2-50f4-47b7-aeb9-b9ee6fc7d49b None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:29.325490 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d646e2-50f4-47b7-aeb9-b9ee6fc7d49b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:29.337492 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d646e2-50f4-47b7-aeb9-b9ee6fc7d49b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 00:59:29.343141 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-73d646e2-50f4-47b7-aeb9-b9ee6fc7d49b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 00:59:29.343141 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 00:59:29.343141 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 00:59:29.343141 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 00:59:29.343141 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 00:59:29.343141 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 00:59:29.343141 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 00:59:29.343141 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 00:59:29.343141 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 00:59:29.343141 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 00:59:29.343141 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 00:59:29.343141 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 00:59:29.343141 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 00:59:29.343141 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 00:59:29.343141 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 00:59:29.343141 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 00:59:29.344069 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 00:59:29.344069 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 00:59:29.344069 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 00:59:29.344069 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 00:59:29.344069 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 00:59:29.344069 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 00:59:29.344069 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 00:59:29.344069 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 00:59:29.344069 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 00:59:29.344069 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 00:59:29.344069 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 00:59:29.344069 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 00:59:29.344069 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 00:59:29.344069 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 00:59:29.344069 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 00:59:29.344069 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 00:59:29.344843 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 00:59:29.344843 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 00:59:29.344843 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 00:59:29.344843 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 00:59:29.344843 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 00:59:29.344843 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 00:59:29.344843 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 00:59:29.344843 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 00:59:29.344843 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 00:59:29.344843 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 00:59:29.344843 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 00:59:29.344843 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 00:59:29.344843 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 53/102] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 8 00:59:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 00:59:29.350190 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16de549-6fdf-4ec1-afed-a72056c8a95b None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:29.403620 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:29.480287 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16de549-6fdf-4ec1-afed-a72056c8a95b None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:29.505983 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a640b-d7a9-4276-b5f9-620b459c8806 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:29.506485 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a640b-d7a9-4276-b5f9-620b459c8806 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:29.506586 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a640b-d7a9-4276-b5f9-620b459c8806 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:29.507856 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 54/103] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 00:59:29.514661 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69494e9-8071-4a4c-9b22-206548f9a64d None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:29.515064 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69494e9-8071-4a4c-9b22-206548f9a64d None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:29.515557 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69494e9-8071-4a4c-9b22-206548f9a64d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:29.531001 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c616824-bae0-4dad-8e15-c25b7f35573b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:29.531631 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c616824-bae0-4dad-8e15-c25b7f35573b None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:29.531915 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c616824-bae0-4dad-8e15-c25b7f35573b None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:29.532209 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c616824-bae0-4dad-8e15-c25b7f35573b None admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:29.532910 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c616824-bae0-4dad-8e15-c25b7f35573b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:29.533193 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c616824-bae0-4dad-8e15-c25b7f35573b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:29.534672 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c616824-bae0-4dad-8e15-c25b7f35573b None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:29.537882 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-b69494e9-8071-4a4c-9b22-206548f9a64d None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:29.539293 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-b69494e9-8071-4a4c-9b22-206548f9a64d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:29.548990 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 50/104] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 00:59:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:29.549990 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c616824-bae0-4dad-8e15-c25b7f35573b None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:29.626981 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 55/105] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 00:59:29.628265 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69494e9-8071-4a4c-9b22-206548f9a64d None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:29.678569 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:29.747264 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69494e9-8071-4a4c-9b22-206548f9a64d None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:29.749551 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:29.752730 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c616824-bae0-4dad-8e15-c25b7f35573b None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:29.804273 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b479665-bd61-422d-8239-39f79fd52daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:29.804910 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b479665-bd61-422d-8239-39f79fd52daa None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:29.805190 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b479665-bd61-422d-8239-39f79fd52daa None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:29.805466 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b479665-bd61-422d-8239-39f79fd52daa None admin] PATH_INFO: `/v3/roles/member` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:29.820609 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b479665-bd61-422d-8239-39f79fd52daa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 00:59:29.826505 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-3b479665-bd61-422d-8239-39f79fd52daa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 00:59:29.826505 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 00:59:29.826505 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 00:59:29.826505 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 00:59:29.826505 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 00:59:29.826505 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 00:59:29.826505 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 00:59:29.826505 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 00:59:29.826505 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 00:59:29.826505 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 00:59:29.826505 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 00:59:29.826505 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 00:59:29.826505 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 00:59:29.826505 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 00:59:29.826505 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 00:59:29.826505 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 00:59:29.827383 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 00:59:29.827383 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 00:59:29.827383 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 00:59:29.827383 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 00:59:29.827383 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 00:59:29.827383 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 00:59:29.827383 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 00:59:29.827383 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 00:59:29.827383 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 00:59:29.827383 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 00:59:29.827383 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 00:59:29.827383 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 00:59:29.827383 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 00:59:29.827383 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 00:59:29.827383 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 00:59:29.827383 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 00:59:29.828148 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 00:59:29.828148 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 00:59:29.828148 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 00:59:29.828148 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 00:59:29.828148 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 00:59:29.828148 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 00:59:29.828148 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 00:59:29.828148 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 00:59:29.828148 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 00:59:29.828148 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 00:59:29.828148 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 00:59:29.828148 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 00:59:29.828762 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f5419-7349-42e1-ab34-d71bc6c6a2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:29.828846 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 51/106] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 00:59:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 00:59:29.828917 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f5419-7349-42e1-ab34-d71bc6c6a2d4 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:29.828917 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f5419-7349-42e1-ab34-d71bc6c6a2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:29.828917 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f5419-7349-42e1-ab34-d71bc6c6a2d4 None admin] PATH_INFO: `/v3/users/51f97eb5e7894117a81b5bd04a11488e` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:29.829866 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1f5419-7349-42e1-ab34-d71bc6c6a2d4 None admin] RBAC: Authorizing `identity:get_user(user_id=51f97eb5e7894117a81b5bd04a11488e)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:29.831090 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1f5419-7349-42e1-ab34-d71bc6c6a2d4 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:29.832843 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 56/107] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 8 00:59:29 2022] GET /identity/v3/users/51f97eb5e7894117a81b5bd04a11488e => generated 247 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:29.835007 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b479665-bd61-422d-8239-39f79fd52daa None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:29.841238 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f5419-7349-42e1-ab34-d71bc6c6a2d4 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:30.032594 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b479665-bd61-422d-8239-39f79fd52daa None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:30.056598 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f5419-7349-42e1-ab34-d71bc6c6a2d4 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:30.072749 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:30.075732 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19244c9a-d3c8-4b1a-9573-8c424fc3fd6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:30.077234 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19244c9a-d3c8-4b1a-9573-8c424fc3fd6d None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:30.079342 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19244c9a-d3c8-4b1a-9573-8c424fc3fd6d None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:30.079617 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19244c9a-d3c8-4b1a-9573-8c424fc3fd6d None admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:30.080317 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19244c9a-d3c8-4b1a-9573-8c424fc3fd6d None admin] RBAC: Adding query filter params (name=member) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:30.080624 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19244c9a-d3c8-4b1a-9573-8c424fc3fd6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:30.082021 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19244c9a-d3c8-4b1a-9573-8c424fc3fd6d None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:30.099224 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 52/108] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 8 00:59:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:30.105118 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19244c9a-d3c8-4b1a-9573-8c424fc3fd6d None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:30.133176 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b85fe0-5b37-4e4a-8b0f-b306f27ea6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:30.133851 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b85fe0-5b37-4e4a-8b0f-b306f27ea6e2 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:30.134165 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b85fe0-5b37-4e4a-8b0f-b306f27ea6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:30.134749 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b85fe0-5b37-4e4a-8b0f-b306f27ea6e2 None admin] PATH_INFO: `/v3/projects/41f15a3cf54f4397920b0e8f758539ea` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:30.135888 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b85fe0-5b37-4e4a-8b0f-b306f27ea6e2 None admin] RBAC: Authorizing `identity:get_project(project_id=41f15a3cf54f4397920b0e8f758539ea)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:30.139757 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b85fe0-5b37-4e4a-8b0f-b306f27ea6e2 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:30.141730 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 57/109] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 00:59:29 2022] GET /identity/v3/projects/41f15a3cf54f4397920b0e8f758539ea => generated 298 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:30.142330 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:30.143886 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472baaf-635d-4e9e-a518-de4c0e9cfa21 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:30.144935 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472baaf-635d-4e9e-a518-de4c0e9cfa21 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:30.145539 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472baaf-635d-4e9e-a518-de4c0e9cfa21 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:30.146975 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 58/110] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 00:59:30.147800 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8472baaf-635d-4e9e-a518-de4c0e9cfa21 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:30.229077 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19244c9a-d3c8-4b1a-9573-8c424fc3fd6d None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:30.254388 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8472baaf-635d-4e9e-a518-de4c0e9cfa21 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:30.267494 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c6d392-94e3-4b9b-8b5b-d155c46c674f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:30.268280 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6d392-94e3-4b9b-8b5b-d155c46c674f None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:30.268570 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6d392-94e3-4b9b-8b5b-d155c46c674f None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:30.268966 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6d392-94e3-4b9b-8b5b-d155c46c674f None admin] PATH_INFO: `/v3/users/3c115932304948d4968c4f07fae9c96b` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:30.270112 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c6d392-94e3-4b9b-8b5b-d155c46c674f None admin] RBAC: Authorizing `identity:get_user(user_id=3c115932304948d4968c4f07fae9c96b)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:30.271774 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c6d392-94e3-4b9b-8b5b-d155c46c674f None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:30.280966 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 53/111] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 8 00:59:30 2022] GET /identity/v3/users/3c115932304948d4968c4f07fae9c96b => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:30.280966 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a66ded-1398-4023-8f20-5c533e621884 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:30.280966 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a66ded-1398-4023-8f20-5c533e621884 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:30.280966 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a66ded-1398-4023-8f20-5c533e621884 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:30.280966 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 54/112] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 00:59:30.280966 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a66ded-1398-4023-8f20-5c533e621884 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:30.297765 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7d0cad-e35f-47a9-a358-bc51fa43c3fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:30.298473 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d0cad-e35f-47a9-a358-bc51fa43c3fd None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:30.298753 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d0cad-e35f-47a9-a358-bc51fa43c3fd None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:30.299024 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d0cad-e35f-47a9-a358-bc51fa43c3fd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:30.309045 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7d0cad-e35f-47a9-a358-bc51fa43c3fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 00:59:30.314156 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-ec7d0cad-e35f-47a9-a358-bc51fa43c3fd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 00:59:30.314156 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 00:59:30.314156 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 00:59:30.314156 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 00:59:30.314156 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 00:59:30.314156 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 00:59:30.314156 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 00:59:30.314156 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 00:59:30.314156 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 00:59:30.314156 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 00:59:30.314156 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 00:59:30.314156 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 00:59:30.314156 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 00:59:30.314156 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 00:59:30.314156 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 00:59:30.314156 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 00:59:30.315000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 00:59:30.315000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 00:59:30.315000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 00:59:30.315000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 00:59:30.315000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 00:59:30.315000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 00:59:30.315000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 00:59:30.315000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 00:59:30.315000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 00:59:30.315000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 00:59:30.315000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 00:59:30.315000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 00:59:30.315000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 00:59:30.315000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 00:59:30.315000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 00:59:30.315000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 00:59:30.315779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 00:59:30.315779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 00:59:30.315779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 00:59:30.315779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 00:59:30.315779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 00:59:30.315779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 00:59:30.315779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 00:59:30.315779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 00:59:30.315779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 00:59:30.315779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 00:59:30.315779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 00:59:30.315779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 00:59:30.315779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 59/113] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 8 00:59:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 00:59:30.317736 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08bcb03-d9b5-4d42-a940-1ce35ceaad9f None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:30.318018 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08bcb03-d9b5-4d42-a940-1ce35ceaad9f None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:30.318291 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08bcb03-d9b5-4d42-a940-1ce35ceaad9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:30.336030 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-e08bcb03-d9b5-4d42-a940-1ce35ceaad9f None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:30.337498 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-e08bcb03-d9b5-4d42-a940-1ce35ceaad9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:30.367731 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:30.380084 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:30.391286 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a66ded-1398-4023-8f20-5c533e621884 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:30.405877 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 60/114] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 00:59:30.406670 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08bcb03-d9b5-4d42-a940-1ce35ceaad9f None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:30.426220 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcd64e7-bbde-489a-ac1e-24346b9d782c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:30.426887 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd64e7-bbde-489a-ac1e-24346b9d782c None admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:30.427175 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd64e7-bbde-489a-ac1e-24346b9d782c None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:30.427454 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd64e7-bbde-489a-ac1e-24346b9d782c None admin] PATH_INFO: `/v3/projects/41f15a3cf54f4397920b0e8f758539ea/users/51f97eb5e7894117a81b5bd04a11488e/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:30.429348 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcd64e7-bbde-489a-ac1e-24346b9d782c None admin] RBAC: Authorizing `identity:create_grant(project_id=41f15a3cf54f4397920b0e8f758539ea, user_id=51f97eb5e7894117a81b5bd04a11488e, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:30.430809 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcd64e7-bbde-489a-ac1e-24346b9d782c None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:30.445402 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 55/115] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 8 00:59:30 2022] PUT /identity/v3/projects/41f15a3cf54f4397920b0e8f758539ea/users/51f97eb5e7894117a81b5bd04a11488e/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 00:59:30.458566 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca42df1-db6b-479e-874f-1c350be2b0cc None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:30.459558 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca42df1-db6b-479e-874f-1c350be2b0cc None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:30.459998 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca42df1-db6b-479e-874f-1c350be2b0cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:30.490651 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-eca42df1-db6b-479e-874f-1c350be2b0cc None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:30.493817 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-eca42df1-db6b-479e-874f-1c350be2b0cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:30.525560 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08bcb03-d9b5-4d42-a940-1ce35ceaad9f None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:30.568623 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2521dee4-654e-4ac7-a465-c944b06f35df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:30.569202 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2521dee4-654e-4ac7-a465-c944b06f35df None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:30.569492 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2521dee4-654e-4ac7-a465-c944b06f35df None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:30.569773 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2521dee4-654e-4ac7-a465-c944b06f35df None admin] PATH_INFO: `/v3/projects/41f15a3cf54f4397920b0e8f758539ea` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:30.570969 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2521dee4-654e-4ac7-a465-c944b06f35df None admin] RBAC: Authorizing `identity:get_project(project_id=41f15a3cf54f4397920b0e8f758539ea)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:30.572510 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2521dee4-654e-4ac7-a465-c944b06f35df None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:30.574246 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 61/116] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 00:59:30 2022] GET /identity/v3/projects/41f15a3cf54f4397920b0e8f758539ea => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:30.581201 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f1d8e-d3c6-4caa-9a08-0b88dad0ff1d None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:30.581476 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f1d8e-d3c6-4caa-9a08-0b88dad0ff1d None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:30.581759 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f1d8e-d3c6-4caa-9a08-0b88dad0ff1d None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:30.583319 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 62/117] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:30 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 00:59:30.583852 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f1d8e-d3c6-4caa-9a08-0b88dad0ff1d None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:30.603773 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 56/118] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 00:59:30.605220 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca42df1-db6b-479e-874f-1c350be2b0cc None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:30.663606 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:30.701999 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:30.727119 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca42df1-db6b-479e-874f-1c350be2b0cc None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:30.750912 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f1d8e-d3c6-4caa-9a08-0b88dad0ff1d None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:30.764460 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76150bb-1c54-4e23-8012-407902392066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:30.765491 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76150bb-1c54-4e23-8012-407902392066 None admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:30.765968 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76150bb-1c54-4e23-8012-407902392066 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:30.766433 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76150bb-1c54-4e23-8012-407902392066 None admin] PATH_INFO: `/v3/projects/41f15a3cf54f4397920b0e8f758539ea/users/3c115932304948d4968c4f07fae9c96b/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:30.768933 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76150bb-1c54-4e23-8012-407902392066 None admin] RBAC: Authorizing `identity:create_grant(project_id=41f15a3cf54f4397920b0e8f758539ea, user_id=3c115932304948d4968c4f07fae9c96b, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:30.770734 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76150bb-1c54-4e23-8012-407902392066 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:30.786834 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 57/119] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 8 00:59:30 2022] PUT /identity/v3/projects/41f15a3cf54f4397920b0e8f758539ea/users/3c115932304948d4968c4f07fae9c96b/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 00:59:30.787989 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ceb8e-928f-405e-b31c-c871fa021ece None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:30.788316 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ceb8e-928f-405e-b31c-c871fa021ece None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:30.788588 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ceb8e-928f-405e-b31c-c871fa021ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:30.796489 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8447f8-9eaa-41d2-825c-b373f43d8570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:30.797586 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8447f8-9eaa-41d2-825c-b373f43d8570 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:30.798061 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8447f8-9eaa-41d2-825c-b373f43d8570 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:30.798631 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8447f8-9eaa-41d2-825c-b373f43d8570 None admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:30.799538 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8447f8-9eaa-41d2-825c-b373f43d8570 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:30.800116 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8447f8-9eaa-41d2-825c-b373f43d8570 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:30.801836 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8447f8-9eaa-41d2-825c-b373f43d8570 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:30.808090 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-ab8ceb8e-928f-405e-b31c-c871fa021ece None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:30.809347 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-ab8ceb8e-928f-405e-b31c-c871fa021ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:30.811786 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 63/120] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 00:59:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:30.813106 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8447f8-9eaa-41d2-825c-b373f43d8570 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:30.932422 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 58/121] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 00:59:30.933632 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8ceb8e-928f-405e-b31c-c871fa021ece None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:31.000295 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:31.005051 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8447f8-9eaa-41d2-825c-b373f43d8570 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:31.032332 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:31.046789 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de06859d-2a7f-4c09-834d-efd8c6415898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:31.047591 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de06859d-2a7f-4c09-834d-efd8c6415898 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:31.047882 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de06859d-2a7f-4c09-834d-efd8c6415898 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:31.048248 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de06859d-2a7f-4c09-834d-efd8c6415898 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:31.049504 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de06859d-2a7f-4c09-834d-efd8c6415898 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:31.051147 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de06859d-2a7f-4c09-834d-efd8c6415898 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:31.053232 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 64/122] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 00:59:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:31.053855 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de06859d-2a7f-4c09-834d-efd8c6415898 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:31.069249 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8ceb8e-928f-405e-b31c-c871fa021ece None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:31.136072 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecf6830-fde0-4fc5-b88f-fddeb44560cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:31.136743 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf6830-fde0-4fc5-b88f-fddeb44560cc None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:31.137027 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf6830-fde0-4fc5-b88f-fddeb44560cc None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:31.137306 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf6830-fde0-4fc5-b88f-fddeb44560cc None admin] PATH_INFO: `/v3/users/3c115932304948d4968c4f07fae9c96b` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:31.140405 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecf6830-fde0-4fc5-b88f-fddeb44560cc None admin] RBAC: Authorizing `identity:get_user(user_id=3c115932304948d4968c4f07fae9c96b)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:31.142435 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecf6830-fde0-4fc5-b88f-fddeb44560cc None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:31.145241 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 59/123] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 8 00:59:30 2022] GET /identity/v3/users/3c115932304948d4968c4f07fae9c96b => generated 275 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:31.145857 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecf6830-fde0-4fc5-b88f-fddeb44560cc None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:31.218226 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de06859d-2a7f-4c09-834d-efd8c6415898 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:31.258548 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb828870-476c-4e73-bcd3-8d3fa399bf15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:31.259200 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb828870-476c-4e73-bcd3-8d3fa399bf15 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:31.259485 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb828870-476c-4e73-bcd3-8d3fa399bf15 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:31.259767 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb828870-476c-4e73-bcd3-8d3fa399bf15 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:31.278794 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb828870-476c-4e73-bcd3-8d3fa399bf15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 00:59:31.285144 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-cb828870-476c-4e73-bcd3-8d3fa399bf15 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 00:59:31.285144 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 00:59:31.285144 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 00:59:31.285144 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 00:59:31.285144 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 00:59:31.285144 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 00:59:31.285144 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 00:59:31.285144 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 00:59:31.285144 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 00:59:31.285144 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 00:59:31.285144 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 00:59:31.285144 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 00:59:31.285144 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 00:59:31.285144 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 00:59:31.285144 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 00:59:31.285144 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 00:59:31.286002 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 00:59:31.286002 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 00:59:31.286002 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 00:59:31.286002 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 00:59:31.286002 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 00:59:31.286002 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 00:59:31.286002 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 00:59:31.286002 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 00:59:31.286002 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 00:59:31.286002 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 00:59:31.286002 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 00:59:31.286002 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 00:59:31.286002 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 00:59:31.286002 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 00:59:31.286002 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 00:59:31.286002 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 00:59:31.286751 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 00:59:31.286751 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 00:59:31.286751 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 00:59:31.286751 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 00:59:31.286751 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 00:59:31.286751 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 00:59:31.286751 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 00:59:31.286751 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 00:59:31.286751 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 00:59:31.286751 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 00:59:31.286751 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 00:59:31.286751 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 00:59:31.289004 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 65/124] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 00:59:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 00:59:31.289984 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb828870-476c-4e73-bcd3-8d3fa399bf15 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:31.320219 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecf6830-fde0-4fc5-b88f-fddeb44560cc None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:31.321604 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:31.359394 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a291ce71-1cef-4307-b941-271e832d7381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:31.360017 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291ce71-1cef-4307-b941-271e832d7381 None admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:31.360387 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291ce71-1cef-4307-b941-271e832d7381 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:31.360672 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291ce71-1cef-4307-b941-271e832d7381 None admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:31.361528 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a291ce71-1cef-4307-b941-271e832d7381 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:31.363606 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a291ce71-1cef-4307-b941-271e832d7381 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:31.376100 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:31.408163 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 60/125] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Sep 8 00:59:31 2022] POST /identity/v3/projects => generated 302 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 00:59:31.408163 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a291ce71-1cef-4307-b941-271e832d7381 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:31.419130 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb828870-476c-4e73-bcd3-8d3fa399bf15 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:31.521706 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62769012-e48e-4e74-8f88-69c4e3481306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:31.522390 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62769012-e48e-4e74-8f88-69c4e3481306 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:31.522674 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62769012-e48e-4e74-8f88-69c4e3481306 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:31.522953 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62769012-e48e-4e74-8f88-69c4e3481306 None admin] PATH_INFO: `/v3/projects/41f15a3cf54f4397920b0e8f758539ea` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:31.525811 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62769012-e48e-4e74-8f88-69c4e3481306 None admin] RBAC: Authorizing `identity:get_project(project_id=41f15a3cf54f4397920b0e8f758539ea)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:31.527344 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62769012-e48e-4e74-8f88-69c4e3481306 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:31.529538 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 66/126] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 00:59:31 2022] GET /identity/v3/projects/41f15a3cf54f4397920b0e8f758539ea => generated 298 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:31.530418 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62769012-e48e-4e74-8f88-69c4e3481306 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:31.560056 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a291ce71-1cef-4307-b941-271e832d7381 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:31.621596 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6449ac2-82d1-4d9a-b752-f8170d20bf8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:31.622334 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6449ac2-82d1-4d9a-b752-f8170d20bf8c None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:31.622627 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6449ac2-82d1-4d9a-b752-f8170d20bf8c None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:31.622973 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6449ac2-82d1-4d9a-b752-f8170d20bf8c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:31.634904 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6449ac2-82d1-4d9a-b752-f8170d20bf8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 00:59:31.641291 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-e6449ac2-82d1-4d9a-b752-f8170d20bf8c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 00:59:31.641291 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 00:59:31.641291 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 00:59:31.641291 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 00:59:31.641291 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 00:59:31.641291 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 00:59:31.641291 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 00:59:31.641291 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 00:59:31.641291 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 00:59:31.641291 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 00:59:31.641291 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 00:59:31.641291 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 00:59:31.641291 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 00:59:31.641291 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 00:59:31.641291 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 00:59:31.641291 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 00:59:31.642157 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 00:59:31.642157 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 00:59:31.642157 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 00:59:31.642157 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 00:59:31.642157 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 00:59:31.642157 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 00:59:31.642157 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 00:59:31.642157 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 00:59:31.642157 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 00:59:31.642157 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 00:59:31.642157 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 00:59:31.642157 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 00:59:31.642157 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 00:59:31.642157 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 00:59:31.642157 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 00:59:31.642157 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 00:59:31.642908 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 00:59:31.642908 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 00:59:31.642908 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 00:59:31.642908 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 00:59:31.642908 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 00:59:31.642908 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 00:59:31.642908 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 00:59:31.642908 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 00:59:31.642908 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 00:59:31.642908 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 00:59:31.642908 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 00:59:31.642908 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 00:59:31.642908 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 61/127] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 00:59:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 00:59:31.644941 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6449ac2-82d1-4d9a-b752-f8170d20bf8c None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:31.679178 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:31.687635 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62769012-e48e-4e74-8f88-69c4e3481306 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:31.704504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:31.725654 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c5b80-5324-46fb-9d66-11ba6670df61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:31.726282 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c5b80-5324-46fb-9d66-11ba6670df61 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:31.726564 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c5b80-5324-46fb-9d66-11ba6670df61 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:31.726842 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c5b80-5324-46fb-9d66-11ba6670df61 None admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:31.727520 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1c5b80-5324-46fb-9d66-11ba6670df61 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:31.727888 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1c5b80-5324-46fb-9d66-11ba6670df61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:31.729547 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1c5b80-5324-46fb-9d66-11ba6670df61 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:31.739765 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 67/128] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 8 00:59:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:31.740832 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c5b80-5324-46fb-9d66-11ba6670df61 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:31.768847 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6449ac2-82d1-4d9a-b752-f8170d20bf8c None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:31.809676 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7eb180-0402-487c-8280-620f3b552ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:31.811076 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7eb180-0402-487c-8280-620f3b552ee2 None admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:31.811440 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7eb180-0402-487c-8280-620f3b552ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:31.811855 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7eb180-0402-487c-8280-620f3b552ee2 None admin] PATH_INFO: `/v3/projects/41f15a3cf54f4397920b0e8f758539ea/users/3c115932304948d4968c4f07fae9c96b/roles/069fe3e033664493b3def844e3735bc9` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:31.814386 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7eb180-0402-487c-8280-620f3b552ee2 None admin] RBAC: Authorizing `identity:create_grant(project_id=41f15a3cf54f4397920b0e8f758539ea, user_id=3c115932304948d4968c4f07fae9c96b, role_id=069fe3e033664493b3def844e3735bc9)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:31.816055 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7eb180-0402-487c-8280-620f3b552ee2 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:31.832674 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 62/129] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 8 00:59:31 2022] PUT /identity/v3/projects/41f15a3cf54f4397920b0e8f758539ea/users/3c115932304948d4968c4f07fae9c96b/roles/069fe3e033664493b3def844e3735bc9 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 00:59:31.833994 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7eb180-0402-487c-8280-620f3b552ee2 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:31.865825 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c5b80-5324-46fb-9d66-11ba6670df61 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:31.933124 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6461f4d-14aa-40f0-b747-604696367bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:31.933859 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6461f4d-14aa-40f0-b747-604696367bc8 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:31.934147 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6461f4d-14aa-40f0-b747-604696367bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:31.934424 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6461f4d-14aa-40f0-b747-604696367bc8 None admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:31.935220 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6461f4d-14aa-40f0-b747-604696367bc8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:31.935626 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6461f4d-14aa-40f0-b747-604696367bc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:31.937137 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6461f4d-14aa-40f0-b747-604696367bc8 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:31.952063 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 68/130] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 8 00:59:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:31.956010 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7eb180-0402-487c-8280-620f3b552ee2 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:31.958502 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6461f4d-14aa-40f0-b747-604696367bc8 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:31.969474 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:31.996938 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75516330-e45f-4854-811d-15a303059779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:31.997654 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:31.999094 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75516330-e45f-4854-811d-15a303059779 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:31.999094 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75516330-e45f-4854-811d-15a303059779 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:31.999459 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75516330-e45f-4854-811d-15a303059779 None admin] PATH_INFO: `/v3/users/6839266d85444d5da67f883a0eba81f8` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:32.020534 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75516330-e45f-4854-811d-15a303059779 None admin] RBAC: Authorizing `identity:get_user(user_id=6839266d85444d5da67f883a0eba81f8)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:32.022626 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75516330-e45f-4854-811d-15a303059779 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:32.024761 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 63/131] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 8 00:59:31 2022] GET /identity/v3/users/6839266d85444d5da67f883a0eba81f8 => generated 289 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:32.033596 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75516330-e45f-4854-811d-15a303059779 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:32.163162 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75516330-e45f-4854-811d-15a303059779 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:32.207569 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6461f4d-14aa-40f0-b747-604696367bc8 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:32.213970 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b243e17-ea07-46c5-ab0c-6f2552a1794f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:32.214611 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b243e17-ea07-46c5-ab0c-6f2552a1794f None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:32.214927 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b243e17-ea07-46c5-ab0c-6f2552a1794f None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:32.215201 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b243e17-ea07-46c5-ab0c-6f2552a1794f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:32.258566 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:32.259356 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8e063b-7cc6-437f-92f3-4000a65288bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:32.260022 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e063b-7cc6-437f-92f3-4000a65288bd None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:32.261030 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e063b-7cc6-437f-92f3-4000a65288bd None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:32.261309 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e063b-7cc6-437f-92f3-4000a65288bd None admin] PATH_INFO: `/v3/users/3c115932304948d4968c4f07fae9c96b` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:32.262617 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8e063b-7cc6-437f-92f3-4000a65288bd None admin] RBAC: Authorizing `identity:get_user(user_id=3c115932304948d4968c4f07fae9c96b)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:32.264940 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8e063b-7cc6-437f-92f3-4000a65288bd None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:32.267736 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 69/132] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 8 00:59:31 2022] GET /identity/v3/users/3c115932304948d4968c4f07fae9c96b => generated 275 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:32.274484 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8e063b-7cc6-437f-92f3-4000a65288bd None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:32.305415 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b243e17-ea07-46c5-ab0c-6f2552a1794f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:32.306927 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b243e17-ea07-46c5-ab0c-6f2552a1794f None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:32.398259 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:32.413815 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 64/133] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 00:59:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 00:59:32.419304 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b243e17-ea07-46c5-ab0c-6f2552a1794f None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:32.420644 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8e063b-7cc6-437f-92f3-4000a65288bd None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:32.468698 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c49940-a997-4515-95d6-33304da036db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:32.469359 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c49940-a997-4515-95d6-33304da036db None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:32.469644 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c49940-a997-4515-95d6-33304da036db None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:32.469922 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c49940-a997-4515-95d6-33304da036db None admin] PATH_INFO: `/v3/projects/34348313aa84446d8c5844c3ed29655b` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:32.476807 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c49940-a997-4515-95d6-33304da036db None admin] RBAC: Authorizing `identity:get_project(project_id=34348313aa84446d8c5844c3ed29655b)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:32.478369 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c49940-a997-4515-95d6-33304da036db None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:32.480375 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 70/134] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 00:59:32 2022] GET /identity/v3/projects/34348313aa84446d8c5844c3ed29655b => generated 312 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 00:59:32.502597 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:32.503336 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c49940-a997-4515-95d6-33304da036db None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:32.542820 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b243e17-ea07-46c5-ab0c-6f2552a1794f None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:32.594745 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b03783-94ff-43f1-a929-394a734c8eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:32.595511 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b03783-94ff-43f1-a929-394a734c8eb1 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:32.595667 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b03783-94ff-43f1-a929-394a734c8eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:32.598016 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b03783-94ff-43f1-a929-394a734c8eb1 None admin] PATH_INFO: `/v3/projects/41f15a3cf54f4397920b0e8f758539ea` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:32.599856 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b03783-94ff-43f1-a929-394a734c8eb1 None admin] RBAC: Authorizing `identity:get_project(project_id=41f15a3cf54f4397920b0e8f758539ea)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:32.601372 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b03783-94ff-43f1-a929-394a734c8eb1 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:32.612308 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 65/135] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 00:59:32 2022] GET /identity/v3/projects/41f15a3cf54f4397920b0e8f758539ea => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 00:59:32.613332 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b03783-94ff-43f1-a929-394a734c8eb1 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:32.756569 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c49940-a997-4515-95d6-33304da036db None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:32.807672 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e304fc3-76c0-429f-9932-57848b7a9936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:32.808364 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e304fc3-76c0-429f-9932-57848b7a9936 None admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:32.808651 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e304fc3-76c0-429f-9932-57848b7a9936 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:32.808930 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e304fc3-76c0-429f-9932-57848b7a9936 None admin] PATH_INFO: `/v3/projects/34348313aa84446d8c5844c3ed29655b/users/3c115932304948d4968c4f07fae9c96b/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:32.810985 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e304fc3-76c0-429f-9932-57848b7a9936 None admin] RBAC: Authorizing `identity:create_grant(project_id=34348313aa84446d8c5844c3ed29655b, user_id=3c115932304948d4968c4f07fae9c96b, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:32.812726 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e304fc3-76c0-429f-9932-57848b7a9936 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:32.829078 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 71/136] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 8 00:59:32 2022] PUT /identity/v3/projects/34348313aa84446d8c5844c3ed29655b/users/3c115932304948d4968c4f07fae9c96b/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 00:59:32.879237 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b03783-94ff-43f1-a929-394a734c8eb1 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:32.932875 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4e5e2b-84d7-4032-910c-6936cb3da764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:32.933501 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e5e2b-84d7-4032-910c-6936cb3da764 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:32.933791 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e5e2b-84d7-4032-910c-6936cb3da764 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:32.934070 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e5e2b-84d7-4032-910c-6936cb3da764 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:32.934821 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4e5e2b-84d7-4032-910c-6936cb3da764 None admin] RBAC: Adding query filter params (role.id=653150c294214a0da3bdd8d00aacd3aa, scope.project.id=41f15a3cf54f4397920b0e8f758539ea, user.id=6839266d85444d5da67f883a0eba81f8) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:32.935136 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4e5e2b-84d7-4032-910c-6936cb3da764 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:32.937424 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4e5e2b-84d7-4032-910c-6936cb3da764 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:32.941576 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:32.945142 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 66/137] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 8 00:59:32 2022] GET /identity/v3/role_assignments?user.id=6839266d85444d5da67f883a0eba81f8&scope.project.id=41f15a3cf54f4397920b0e8f758539ea&role.id=653150c294214a0da3bdd8d00aacd3aa => generated 259 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:33.451263 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a807271-411e-412d-80da-fbf871dd4ec0 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:33.452061 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a807271-411e-412d-80da-fbf871dd4ec0 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:33.452593 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a807271-411e-412d-80da-fbf871dd4ec0 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:33.454146 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 72/138] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 00:59:33.459388 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb3afc-609e-45e1-9ec8-c23600c49870 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:33.459678 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb3afc-609e-45e1-9ec8-c23600c49870 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:33.459956 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb3afc-609e-45e1-9ec8-c23600c49870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:33.479059 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-49fb3afc-609e-45e1-9ec8-c23600c49870 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:33.480283 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-49fb3afc-609e-45e1-9ec8-c23600c49870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:33.569050 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 67/139] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 00:59:33.834258 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b0ae44-4c8f-4f89-a010-a54c38c5634c None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:33.834792 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b0ae44-4c8f-4f89-a010-a54c38c5634c None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:33.834792 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b0ae44-4c8f-4f89-a010-a54c38c5634c None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:33.836379 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 73/140] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 00:59:33.842413 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463114e3-f5ea-4f07-b750-9c3a540d06c6 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:33.842960 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463114e3-f5ea-4f07-b750-9c3a540d06c6 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:33.843354 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463114e3-f5ea-4f07-b750-9c3a540d06c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:33.862532 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-463114e3-f5ea-4f07-b750-9c3a540d06c6 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:33.863732 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-463114e3-f5ea-4f07-b750-9c3a540d06c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:33.937007 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 68/141] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 00:59:34.109453 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b0ae44-4c8f-4f89-a010-a54c38c5634c None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:34.126038 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:34.216368 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b0ae44-4c8f-4f89-a010-a54c38c5634c None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:34.251181 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce3f306-846b-4605-a08b-fca6f19704b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:34.251817 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce3f306-846b-4605-a08b-fca6f19704b0 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:34.252093 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce3f306-846b-4605-a08b-fca6f19704b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:34.252400 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce3f306-846b-4605-a08b-fca6f19704b0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:34.254783 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463114e3-f5ea-4f07-b750-9c3a540d06c6 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:34.268162 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce3f306-846b-4605-a08b-fca6f19704b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 00:59:34.268934 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-fce3f306-846b-4605-a08b-fca6f19704b0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 00:59:34.268934 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 00:59:34.268934 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 00:59:34.268934 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 00:59:34.268934 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 00:59:34.268934 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 00:59:34.268934 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 00:59:34.268934 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 00:59:34.268934 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 00:59:34.268934 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 00:59:34.268934 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 00:59:34.268934 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 00:59:34.268934 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 00:59:34.268934 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 00:59:34.268934 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 00:59:34.268934 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 00:59:34.269767 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 00:59:34.269767 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 00:59:34.269767 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 00:59:34.269767 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 00:59:34.269767 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 00:59:34.269767 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 00:59:34.269767 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 00:59:34.269767 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 00:59:34.269767 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 00:59:34.269767 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 00:59:34.269767 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 00:59:34.269767 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 00:59:34.269767 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 00:59:34.269767 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 00:59:34.269767 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 00:59:34.269767 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 00:59:34.270511 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 00:59:34.270511 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 00:59:34.270511 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 00:59:34.270511 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 00:59:34.270511 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 00:59:34.270511 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 00:59:34.270511 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 00:59:34.270511 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 00:59:34.270511 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 00:59:34.270511 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 00:59:34.270511 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 00:59:34.270511 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 00:59:34.270511 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 74/142] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 8 00:59:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 00:59:34.275940 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce3f306-846b-4605-a08b-fca6f19704b0 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:34.364613 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:34.416481 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:34.436834 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463114e3-f5ea-4f07-b750-9c3a540d06c6 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:34.460108 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce3f306-846b-4605-a08b-fca6f19704b0 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:34.500073 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21809292-cdb9-438b-a1b7-b623a50426e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:34.500789 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21809292-cdb9-438b-a1b7-b623a50426e5 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:34.501073 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21809292-cdb9-438b-a1b7-b623a50426e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:34.501341 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21809292-cdb9-438b-a1b7-b623a50426e5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:34.519642 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21809292-cdb9-438b-a1b7-b623a50426e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 00:59:34.523039 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceaffb0-1757-4b97-9f67-56627b10779e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:34.523647 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceaffb0-1757-4b97-9f67-56627b10779e None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:34.523924 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceaffb0-1757-4b97-9f67-56627b10779e None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:34.524260 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceaffb0-1757-4b97-9f67-56627b10779e None admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:34.524974 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceaffb0-1757-4b97-9f67-56627b10779e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:34.525278 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceaffb0-1757-4b97-9f67-56627b10779e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:34.526647 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceaffb0-1757-4b97-9f67-56627b10779e None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:34.535078 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-21809292-cdb9-438b-a1b7-b623a50426e5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 00:59:34.535078 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 00:59:34.535078 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 00:59:34.535078 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 00:59:34.535078 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 00:59:34.535078 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 00:59:34.535078 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 00:59:34.535078 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 00:59:34.535078 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 00:59:34.535078 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 00:59:34.535078 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 00:59:34.535078 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 00:59:34.535078 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 00:59:34.535078 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 00:59:34.535078 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 00:59:34.535078 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 00:59:34.535955 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 00:59:34.535955 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 00:59:34.535955 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 00:59:34.535955 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 00:59:34.535955 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 00:59:34.535955 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 00:59:34.535955 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 00:59:34.535955 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 00:59:34.535955 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 00:59:34.535955 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 00:59:34.535955 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 00:59:34.535955 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 00:59:34.535955 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 00:59:34.535955 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 00:59:34.535955 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 00:59:34.535955 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 00:59:34.536747 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 00:59:34.536747 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 00:59:34.536747 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 00:59:34.536747 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 00:59:34.536747 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 00:59:34.536747 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 00:59:34.536747 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 00:59:34.536747 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 00:59:34.536747 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 00:59:34.536747 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 00:59:34.536747 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 00:59:34.536747 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 00:59:34.536747 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 69/143] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 00:59:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 00:59:34.542105 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21809292-cdb9-438b-a1b7-b623a50426e5 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:34.543139 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 75/144] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 00:59:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 00:59:34.553134 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceaffb0-1757-4b97-9f67-56627b10779e None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:34.734806 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21809292-cdb9-438b-a1b7-b623a50426e5 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:34.745698 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceaffb0-1757-4b97-9f67-56627b10779e None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:34.787797 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce79f57-83e2-4336-be53-3747533c1fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:34.788473 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce79f57-83e2-4336-be53-3747533c1fc7 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:34.788661 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:34.788766 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce79f57-83e2-4336-be53-3747533c1fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:34.789005 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce79f57-83e2-4336-be53-3747533c1fc7 None admin] PATH_INFO: `/v3/users/51f97eb5e7894117a81b5bd04a11488e` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:34.790575 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce79f57-83e2-4336-be53-3747533c1fc7 None admin] RBAC: Authorizing `identity:get_user(user_id=51f97eb5e7894117a81b5bd04a11488e)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:34.792037 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce79f57-83e2-4336-be53-3747533c1fc7 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:34.796790 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 76/145] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 8 00:59:34 2022] GET /identity/v3/users/51f97eb5e7894117a81b5bd04a11488e => generated 247 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 00:59:34.802116 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce79f57-83e2-4336-be53-3747533c1fc7 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:34.807257 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:34.823431 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fdf585-3d0b-4539-8c6c-3321f84f2a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:34.824161 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fdf585-3d0b-4539-8c6c-3321f84f2a40 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:34.824520 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fdf585-3d0b-4539-8c6c-3321f84f2a40 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:34.824864 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fdf585-3d0b-4539-8c6c-3321f84f2a40 None admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:34.825610 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fdf585-3d0b-4539-8c6c-3321f84f2a40 None admin] RBAC: Adding query filter params (name=member) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:34.826108 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fdf585-3d0b-4539-8c6c-3321f84f2a40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:34.830159 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fdf585-3d0b-4539-8c6c-3321f84f2a40 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:34.843043 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 70/146] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 8 00:59:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:34.849722 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fdf585-3d0b-4539-8c6c-3321f84f2a40 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:35.010556 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce79f57-83e2-4336-be53-3747533c1fc7 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:35.061465 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fdf585-3d0b-4539-8c6c-3321f84f2a40 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:35.085794 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a8f581-e3c3-4066-98db-1a4d27ab7e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:35.086407 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8f581-e3c3-4066-98db-1a4d27ab7e9f None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:35.086725 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8f581-e3c3-4066-98db-1a4d27ab7e9f None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:35.086999 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8f581-e3c3-4066-98db-1a4d27ab7e9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:35.127028 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f061f7-a819-4da7-ae6c-49a4edcc90a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:35.127673 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f061f7-a819-4da7-ae6c-49a4edcc90a9 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:35.127948 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f061f7-a819-4da7-ae6c-49a4edcc90a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:35.128238 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f061f7-a819-4da7-ae6c-49a4edcc90a9 None admin] PATH_INFO: `/v3/users/3c115932304948d4968c4f07fae9c96b` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:35.129397 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f061f7-a819-4da7-ae6c-49a4edcc90a9 None admin] RBAC: Authorizing `identity:get_user(user_id=3c115932304948d4968c4f07fae9c96b)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:35.130998 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f061f7-a819-4da7-ae6c-49a4edcc90a9 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:35.132628 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 71/147] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 8 00:59:34 2022] GET /identity/v3/users/3c115932304948d4968c4f07fae9c96b => generated 275 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:35.138278 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f061f7-a819-4da7-ae6c-49a4edcc90a9 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:35.188176 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a8f581-e3c3-4066-98db-1a4d27ab7e9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:35.189890 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a8f581-e3c3-4066-98db-1a4d27ab7e9f None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:35.192758 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:35.201370 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:35.251677 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f061f7-a819-4da7-ae6c-49a4edcc90a9 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:35.289118 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7093079-b291-46c9-b6ec-26f68a4c97b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:35.289644 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7093079-b291-46c9-b6ec-26f68a4c97b4 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:35.289921 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7093079-b291-46c9-b6ec-26f68a4c97b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:35.290192 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7093079-b291-46c9-b6ec-26f68a4c97b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:35.320354 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 77/148] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 00:59:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 08 00:59:35.321927 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f6706-76ee-4aa7-a127-0ec3b8374140 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:35.322427 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f6706-76ee-4aa7-a127-0ec3b8374140 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:35.323099 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f6706-76ee-4aa7-a127-0ec3b8374140 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:35.325438 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 78/149] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 00:59:35.327204 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e7d7ad-e745-4326-855e-a96fe347505b None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:35.327899 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e7d7ad-e745-4326-855e-a96fe347505b None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:35.328406 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e7d7ad-e745-4326-855e-a96fe347505b None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:35.330076 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 79/150] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 00:59:35.335219 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e7d7ad-e745-4326-855e-a96fe347505b None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:35.366451 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7093079-b291-46c9-b6ec-26f68a4c97b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:35.368384 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7093079-b291-46c9-b6ec-26f68a4c97b4 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:35.424019 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:35.471173 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e7d7ad-e745-4326-855e-a96fe347505b None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:35.474410 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:35.492996 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 72/151] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 00:59:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 00:59:35.494722 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fafe27-4cb4-495e-9d38-53d7bb38ac41 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:35.495011 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fafe27-4cb4-495e-9d38-53d7bb38ac41 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:35.495307 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fafe27-4cb4-495e-9d38-53d7bb38ac41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:35.518774 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943effe8-77fe-41cd-b9aa-09a436d9103c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:35.519790 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943effe8-77fe-41cd-b9aa-09a436d9103c None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:35.520099 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-80fafe27-4cb4-495e-9d38-53d7bb38ac41 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:35.520487 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943effe8-77fe-41cd-b9aa-09a436d9103c None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:35.520929 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943effe8-77fe-41cd-b9aa-09a436d9103c None admin] PATH_INFO: `/v3/projects/41f15a3cf54f4397920b0e8f758539ea` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:35.521635 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-80fafe27-4cb4-495e-9d38-53d7bb38ac41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:35.522394 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943effe8-77fe-41cd-b9aa-09a436d9103c None admin] RBAC: Authorizing `identity:get_project(project_id=41f15a3cf54f4397920b0e8f758539ea)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:35.524339 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943effe8-77fe-41cd-b9aa-09a436d9103c None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:35.527051 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 80/152] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 00:59:35 2022] GET /identity/v3/projects/41f15a3cf54f4397920b0e8f758539ea => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:35.528462 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987e931-ccfa-4393-86b2-fc6e7f636bee None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:35.528918 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987e931-ccfa-4393-86b2-fc6e7f636bee None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:35.529919 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987e931-ccfa-4393-86b2-fc6e7f636bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:35.553305 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-9987e931-ccfa-4393-86b2-fc6e7f636bee None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:35.555635 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-9987e931-ccfa-4393-86b2-fc6e7f636bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:35.610139 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 73/153] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 00:59:35.611718 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fafe27-4cb4-495e-9d38-53d7bb38ac41 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:35.669455 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 81/154] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 00:59:35.670543 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9987e931-ccfa-4393-86b2-fc6e7f636bee None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:35.732102 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:35.774272 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fafe27-4cb4-495e-9d38-53d7bb38ac41 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:35.810673 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a4cc7-16d3-450e-a816-18ca512dc172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:35.811799 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a4cc7-16d3-450e-a816-18ca512dc172 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:35.812398 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a4cc7-16d3-450e-a816-18ca512dc172 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:35.813013 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a4cc7-16d3-450e-a816-18ca512dc172 None admin] PATH_INFO: `/v3/projects/41f15a3cf54f4397920b0e8f758539ea` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:35.814409 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779a4cc7-16d3-450e-a816-18ca512dc172 None admin] RBAC: Authorizing `identity:get_project(project_id=41f15a3cf54f4397920b0e8f758539ea)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:35.816069 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779a4cc7-16d3-450e-a816-18ca512dc172 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:35.817935 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 74/155] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 00:59:35 2022] GET /identity/v3/projects/41f15a3cf54f4397920b0e8f758539ea => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:35.830515 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9987e931-ccfa-4393-86b2-fc6e7f636bee None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:35.832031 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:35.835340 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a4cc7-16d3-450e-a816-18ca512dc172 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:35.884954 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc7459-ca76-4d84-bf47-e2f4d5e5a350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:35.886485 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc7459-ca76-4d84-bf47-e2f4d5e5a350 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:35.886485 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc7459-ca76-4d84-bf47-e2f4d5e5a350 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:35.886485 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc7459-ca76-4d84-bf47-e2f4d5e5a350 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:35.887222 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befc7459-ca76-4d84-bf47-e2f4d5e5a350 None admin] RBAC: Adding query filter params (role.id=34de369ec7c54715aa83a9052a26c3f2, scope.project.id=41f15a3cf54f4397920b0e8f758539ea, user.id=51f97eb5e7894117a81b5bd04a11488e) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:35.887522 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befc7459-ca76-4d84-bf47-e2f4d5e5a350 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:35.889583 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befc7459-ca76-4d84-bf47-e2f4d5e5a350 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:35.904663 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 82/156] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 8 00:59:35 2022] GET /identity/v3/role_assignments?user.id=51f97eb5e7894117a81b5bd04a11488e&scope.project.id=41f15a3cf54f4397920b0e8f758539ea&role.id=34de369ec7c54715aa83a9052a26c3f2 => generated 608 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:35.905900 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b153433-d4a8-48ea-9192-0985713414f2 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:35.906176 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b153433-d4a8-48ea-9192-0985713414f2 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:35.906446 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b153433-d4a8-48ea-9192-0985713414f2 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:35.908055 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 83/157] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 00:59:35.915083 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5033bd9-22d9-417e-bf1a-c0ef1973abdf None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:35.915459 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5033bd9-22d9-417e-bf1a-c0ef1973abdf None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:35.915743 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5033bd9-22d9-417e-bf1a-c0ef1973abdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:35.950582 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-f5033bd9-22d9-417e-bf1a-c0ef1973abdf None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:35.953000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-f5033bd9-22d9-417e-bf1a-c0ef1973abdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:35.971298 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a4cc7-16d3-450e-a816-18ca512dc172 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:36.011997 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3dd941-f9a7-476c-a9b0-2f5faa19a4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:36.013106 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3dd941-f9a7-476c-a9b0-2f5faa19a4f4 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:36.014165 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3dd941-f9a7-476c-a9b0-2f5faa19a4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:36.014638 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3dd941-f9a7-476c-a9b0-2f5faa19a4f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:36.015611 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3dd941-f9a7-476c-a9b0-2f5faa19a4f4 None admin] RBAC: Adding query filter params (role.id=92aa6d6824404d10b4a3be8b4a6bc317, scope.project.id=41f15a3cf54f4397920b0e8f758539ea, user.id=3c115932304948d4968c4f07fae9c96b) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:36.016128 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3dd941-f9a7-476c-a9b0-2f5faa19a4f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:36.017873 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3dd941-f9a7-476c-a9b0-2f5faa19a4f4 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:36.025063 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 75/158] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 8 00:59:35 2022] GET /identity/v3/role_assignments?user.id=3c115932304948d4968c4f07fae9c96b&scope.project.id=41f15a3cf54f4397920b0e8f758539ea&role.id=92aa6d6824404d10b4a3be8b4a6bc317 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 00:59:36.026267 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3dd941-f9a7-476c-a9b0-2f5faa19a4f4 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:36.045121 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:36.091685 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 84/159] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 00:59:36.171615 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3dd941-f9a7-476c-a9b0-2f5faa19a4f4 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:36.178320 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5033bd9-22d9-417e-bf1a-c0ef1973abdf None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:36.232124 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410262df-b92b-41a1-bcf8-aea4e35281ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:36.232946 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410262df-b92b-41a1-bcf8-aea4e35281ad None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:36.233244 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410262df-b92b-41a1-bcf8-aea4e35281ad None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:36.233599 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410262df-b92b-41a1-bcf8-aea4e35281ad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:36.254587 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410262df-b92b-41a1-bcf8-aea4e35281ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 00:59:36.259876 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-410262df-b92b-41a1-bcf8-aea4e35281ad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 00:59:36.259876 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 00:59:36.259876 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 00:59:36.259876 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 00:59:36.259876 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 00:59:36.259876 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 00:59:36.259876 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 00:59:36.259876 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 00:59:36.259876 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 00:59:36.259876 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 00:59:36.259876 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 00:59:36.259876 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 00:59:36.259876 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 00:59:36.259876 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 00:59:36.259876 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 00:59:36.259876 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 00:59:36.261090 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 00:59:36.261090 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 00:59:36.261090 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 00:59:36.261090 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 00:59:36.261090 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 00:59:36.261090 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 00:59:36.261090 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 00:59:36.261090 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 00:59:36.261090 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 00:59:36.261090 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 00:59:36.261090 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 00:59:36.261090 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 00:59:36.261090 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 00:59:36.261090 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 00:59:36.261090 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 00:59:36.261090 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 00:59:36.261941 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 00:59:36.261941 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 00:59:36.261941 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 00:59:36.261941 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 00:59:36.261941 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 00:59:36.261941 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 00:59:36.261941 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 00:59:36.261941 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 00:59:36.261941 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 00:59:36.261941 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 00:59:36.261941 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 00:59:36.261941 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 00:59:36.273298 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 76/160] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 8 00:59:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 00:59:36.295053 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410262df-b92b-41a1-bcf8-aea4e35281ad None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:36.307562 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:36.315234 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5033bd9-22d9-417e-bf1a-c0ef1973abdf None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:36.352672 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e865b38-9a45-45f6-ac6f-3ad8134aed79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:36.353235 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e865b38-9a45-45f6-ac6f-3ad8134aed79 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:36.353505 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e865b38-9a45-45f6-ac6f-3ad8134aed79 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:36.353774 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e865b38-9a45-45f6-ac6f-3ad8134aed79 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:36.354897 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e865b38-9a45-45f6-ac6f-3ad8134aed79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:36.356388 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e865b38-9a45-45f6-ac6f-3ad8134aed79 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:36.359108 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 85/161] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 00:59:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 00:59:36.364562 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e865b38-9a45-45f6-ac6f-3ad8134aed79 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:36.396497 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:36.444151 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410262df-b92b-41a1-bcf8-aea4e35281ad None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:36.474286 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e865b38-9a45-45f6-ac6f-3ad8134aed79 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:36.489305 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a8fedd-30be-4253-a92f-9cc1a250fa0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:36.489923 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a8fedd-30be-4253-a92f-9cc1a250fa0d None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:36.490200 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a8fedd-30be-4253-a92f-9cc1a250fa0d None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:36.490484 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a8fedd-30be-4253-a92f-9cc1a250fa0d None admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:36.491135 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a8fedd-30be-4253-a92f-9cc1a250fa0d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:36.491465 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a8fedd-30be-4253-a92f-9cc1a250fa0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:36.493086 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a8fedd-30be-4253-a92f-9cc1a250fa0d None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:36.502708 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 77/162] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 00:59:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 00:59:36.503641 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a8fedd-30be-4253-a92f-9cc1a250fa0d None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:36.518784 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7538af-9a5a-40c1-90ae-c76f7c11f60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:36.519824 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7538af-9a5a-40c1-90ae-c76f7c11f60a None admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:36.520327 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7538af-9a5a-40c1-90ae-c76f7c11f60a None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:36.520801 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7538af-9a5a-40c1-90ae-c76f7c11f60a None admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:36.521681 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7538af-9a5a-40c1-90ae-c76f7c11f60a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:36.523521 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7538af-9a5a-40c1-90ae-c76f7c11f60a None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:36.551791 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 86/163] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 8 00:59:36 2022] POST /identity/v3/users => generated 283 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 00:59:36.553723 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7538af-9a5a-40c1-90ae-c76f7c11f60a None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:36.586288 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:36.644476 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a8fedd-30be-4253-a92f-9cc1a250fa0d None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:36.687005 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67209828-99da-4bf3-bf5a-76454c14fff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:36.687662 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67209828-99da-4bf3-bf5a-76454c14fff4 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:36.687941 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67209828-99da-4bf3-bf5a-76454c14fff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:36.688247 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67209828-99da-4bf3-bf5a-76454c14fff4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:36.688870 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7538af-9a5a-40c1-90ae-c76f7c11f60a None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:36.690780 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:36.707212 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67209828-99da-4bf3-bf5a-76454c14fff4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 00:59:36.713129 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-67209828-99da-4bf3-bf5a-76454c14fff4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 00:59:36.713129 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 00:59:36.713129 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 00:59:36.713129 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 00:59:36.713129 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 00:59:36.713129 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 00:59:36.713129 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 00:59:36.713129 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 00:59:36.713129 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 00:59:36.713129 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 00:59:36.713129 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 00:59:36.713129 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 00:59:36.713129 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 00:59:36.713129 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 00:59:36.713129 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 00:59:36.713129 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 00:59:36.714033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 00:59:36.714033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 00:59:36.714033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 00:59:36.714033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 00:59:36.714033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 00:59:36.714033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 00:59:36.714033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 00:59:36.714033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 00:59:36.714033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 00:59:36.714033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 00:59:36.714033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 00:59:36.714033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 00:59:36.714033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 00:59:36.714033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 00:59:36.714033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 00:59:36.714033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 00:59:36.714805 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 00:59:36.714805 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 00:59:36.714805 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 00:59:36.714805 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 00:59:36.714805 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 00:59:36.714805 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 00:59:36.714805 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 00:59:36.714805 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 00:59:36.714805 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 00:59:36.714805 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 00:59:36.714805 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 00:59:36.714805 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 00:59:36.714805 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 78/164] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 00:59:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 00:59:36.716611 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c9c17-7001-44d0-881d-e7acd394d20a None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:36.716611 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c9c17-7001-44d0-881d-e7acd394d20a None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:36.716611 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c9c17-7001-44d0-881d-e7acd394d20a None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:36.717782 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 79/165] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 00:59:36.722499 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581c9c17-7001-44d0-881d-e7acd394d20a None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:36.734230 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aa4288-257c-483e-95e3-bd31946b4215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:36.734418 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa4288-257c-483e-95e3-bd31946b4215 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:36.734592 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa4288-257c-483e-95e3-bd31946b4215 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:36.734867 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa4288-257c-483e-95e3-bd31946b4215 None admin] PATH_INFO: `/v3/users/3c115932304948d4968c4f07fae9c96b` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:36.736901 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aa4288-257c-483e-95e3-bd31946b4215 None admin] RBAC: Authorizing `identity:get_user(user_id=3c115932304948d4968c4f07fae9c96b)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:36.738406 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aa4288-257c-483e-95e3-bd31946b4215 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:36.740008 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 87/166] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 8 00:59:36 2022] GET /identity/v3/users/3c115932304948d4968c4f07fae9c96b => generated 275 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:36.741213 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676e1ae-ab8b-4ea7-8963-3ebaee57dcbc None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:36.741523 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676e1ae-ab8b-4ea7-8963-3ebaee57dcbc None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:36.741775 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676e1ae-ab8b-4ea7-8963-3ebaee57dcbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:36.769182 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-2676e1ae-ab8b-4ea7-8963-3ebaee57dcbc None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:36.770376 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-2676e1ae-ab8b-4ea7-8963-3ebaee57dcbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:36.855515 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581c9c17-7001-44d0-881d-e7acd394d20a None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:36.861329 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 88/167] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 00:59:36.862185 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2676e1ae-ab8b-4ea7-8963-3ebaee57dcbc None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:36.899792 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f19095-0cb0-495d-9e2c-1e53901e9347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:36.900415 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f19095-0cb0-495d-9e2c-1e53901e9347 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:36.900677 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f19095-0cb0-495d-9e2c-1e53901e9347 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:36.900947 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f19095-0cb0-495d-9e2c-1e53901e9347 None admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:36.901614 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f19095-0cb0-495d-9e2c-1e53901e9347 None admin] RBAC: Adding query filter params (name=member) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:36.901918 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f19095-0cb0-495d-9e2c-1e53901e9347 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:36.903310 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f19095-0cb0-495d-9e2c-1e53901e9347 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:36.913191 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 80/168] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 8 00:59:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:36.913911 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:36.918758 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f19095-0cb0-495d-9e2c-1e53901e9347 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:36.990280 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:37.026965 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2676e1ae-ab8b-4ea7-8963-3ebaee57dcbc None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:37.042151 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f19095-0cb0-495d-9e2c-1e53901e9347 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:37.070147 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e91ea58-0ab6-46e5-aa04-10b21b89dfda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:37.070489 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e91ea58-0ab6-46e5-aa04-10b21b89dfda None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:37.070772 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e91ea58-0ab6-46e5-aa04-10b21b89dfda None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:37.071044 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e91ea58-0ab6-46e5-aa04-10b21b89dfda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:37.089029 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab7d4ee-bf25-4138-8405-3baadf64ad3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:37.089836 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab7d4ee-bf25-4138-8405-3baadf64ad3a None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:37.090122 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab7d4ee-bf25-4138-8405-3baadf64ad3a None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:37.090400 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab7d4ee-bf25-4138-8405-3baadf64ad3a None admin] PATH_INFO: `/v3/users/3c115932304948d4968c4f07fae9c96b` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:37.092547 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab7d4ee-bf25-4138-8405-3baadf64ad3a None admin] RBAC: Authorizing `identity:get_user(user_id=3c115932304948d4968c4f07fae9c96b)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:37.094953 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab7d4ee-bf25-4138-8405-3baadf64ad3a None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:37.096655 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 81/169] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 8 00:59:36 2022] GET /identity/v3/users/3c115932304948d4968c4f07fae9c96b => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 00:59:37.102240 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab7d4ee-bf25-4138-8405-3baadf64ad3a None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:37.159094 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e91ea58-0ab6-46e5-aa04-10b21b89dfda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:37.161506 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e91ea58-0ab6-46e5-aa04-10b21b89dfda None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:37.183061 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:37.235984 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab7d4ee-bf25-4138-8405-3baadf64ad3a None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:37.272745 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:37.280979 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 89/170] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 00:59:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 00:59:37.281917 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e91ea58-0ab6-46e5-aa04-10b21b89dfda None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:37.287908 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41af80d7-0064-495b-93f8-802683f8e67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:37.288564 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af80d7-0064-495b-93f8-802683f8e67a None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:37.288837 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af80d7-0064-495b-93f8-802683f8e67a None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:37.289102 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af80d7-0064-495b-93f8-802683f8e67a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:37.381981 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41af80d7-0064-495b-93f8-802683f8e67a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:37.383467 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41af80d7-0064-495b-93f8-802683f8e67a None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:37.405307 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e91ea58-0ab6-46e5-aa04-10b21b89dfda None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:37.425957 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:37.446163 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c129fc4-95f2-403c-bf9f-060c595bb97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:37.447070 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c129fc4-95f2-403c-bf9f-060c595bb97b None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:37.447495 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c129fc4-95f2-403c-bf9f-060c595bb97b None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:37.447924 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c129fc4-95f2-403c-bf9f-060c595bb97b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:37.460574 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c129fc4-95f2-403c-bf9f-060c595bb97b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 00:59:37.466046 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-4c129fc4-95f2-403c-bf9f-060c595bb97b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 00:59:37.466046 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 00:59:37.466046 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 00:59:37.466046 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 00:59:37.466046 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 00:59:37.466046 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 00:59:37.466046 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 00:59:37.466046 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 00:59:37.466046 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 00:59:37.466046 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 00:59:37.466046 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 00:59:37.466046 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 00:59:37.466046 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 00:59:37.466046 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 00:59:37.466046 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 00:59:37.466046 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 00:59:37.467237 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 00:59:37.467237 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 00:59:37.467237 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 00:59:37.467237 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 00:59:37.467237 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 00:59:37.467237 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 00:59:37.467237 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 00:59:37.467237 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 00:59:37.467237 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 00:59:37.467237 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 00:59:37.467237 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 00:59:37.467237 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 00:59:37.467237 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 00:59:37.467237 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 00:59:37.467237 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 00:59:37.467237 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 00:59:37.468309 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 00:59:37.468309 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 00:59:37.468309 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 00:59:37.468309 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 00:59:37.468309 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 00:59:37.468309 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 00:59:37.468309 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 00:59:37.468309 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 00:59:37.468309 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 00:59:37.468309 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 00:59:37.468309 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 00:59:37.468309 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 00:59:37.470390 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 90/171] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 00:59:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 00:59:37.471406 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c129fc4-95f2-403c-bf9f-060c595bb97b None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:37.498838 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 82/172] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 00:59:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 08 00:59:37.499644 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41af80d7-0064-495b-93f8-802683f8e67a None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:37.525353 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:37.592507 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c129fc4-95f2-403c-bf9f-060c595bb97b None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:37.610295 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41af80d7-0064-495b-93f8-802683f8e67a None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:37.632120 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d5429-2f24-4bc9-9e00-ef3a19235e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:37.633990 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d5429-2f24-4bc9-9e00-ef3a19235e96 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:37.634571 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d5429-2f24-4bc9-9e00-ef3a19235e96 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:37.635088 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d5429-2f24-4bc9-9e00-ef3a19235e96 None admin] PATH_INFO: `/v3/projects/41f15a3cf54f4397920b0e8f758539ea` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:37.636808 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811d5429-2f24-4bc9-9e00-ef3a19235e96 None admin] RBAC: Authorizing `identity:get_project(project_id=41f15a3cf54f4397920b0e8f758539ea)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:37.638658 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811d5429-2f24-4bc9-9e00-ef3a19235e96 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:37.640686 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 91/173] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 00:59:37 2022] GET /identity/v3/projects/41f15a3cf54f4397920b0e8f758539ea => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:37.641833 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d5429-2f24-4bc9-9e00-ef3a19235e96 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:37.653222 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61fa2b9-2a7f-430f-b0a5-9f7dcaaa40e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:37.653834 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61fa2b9-2a7f-430f-b0a5-9f7dcaaa40e9 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:37.654636 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61fa2b9-2a7f-430f-b0a5-9f7dcaaa40e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:37.654902 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61fa2b9-2a7f-430f-b0a5-9f7dcaaa40e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:37.655662 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61fa2b9-2a7f-430f-b0a5-9f7dcaaa40e9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:37.655964 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61fa2b9-2a7f-430f-b0a5-9f7dcaaa40e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:37.657368 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61fa2b9-2a7f-430f-b0a5-9f7dcaaa40e9 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:37.666395 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 83/174] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 8 00:59:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:37.672296 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61fa2b9-2a7f-430f-b0a5-9f7dcaaa40e9 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:37.700269 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:37.759911 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d5429-2f24-4bc9-9e00-ef3a19235e96 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:37.776520 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:37.795051 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61fa2b9-2a7f-430f-b0a5-9f7dcaaa40e9 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:37.800320 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a83215-53bb-4f64-8d34-ce005ba2151a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:37.801098 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a83215-53bb-4f64-8d34-ce005ba2151a None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:37.801476 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a83215-53bb-4f64-8d34-ce005ba2151a None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:37.801849 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a83215-53bb-4f64-8d34-ce005ba2151a None admin] PATH_INFO: `/v3/projects/34348313aa84446d8c5844c3ed29655b` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:37.803250 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a83215-53bb-4f64-8d34-ce005ba2151a None admin] RBAC: Authorizing `identity:get_project(project_id=34348313aa84446d8c5844c3ed29655b)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:37.804972 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a83215-53bb-4f64-8d34-ce005ba2151a None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:37.806834 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 92/175] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 00:59:37 2022] GET /identity/v3/projects/34348313aa84446d8c5844c3ed29655b => generated 312 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:37.807677 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a83215-53bb-4f64-8d34-ce005ba2151a None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:37.833244 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740a9e32-5ffe-43bd-993a-61d531f36b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:37.834130 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a9e32-5ffe-43bd-993a-61d531f36b72 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:37.834547 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a9e32-5ffe-43bd-993a-61d531f36b72 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:37.835055 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a9e32-5ffe-43bd-993a-61d531f36b72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:37.835927 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740a9e32-5ffe-43bd-993a-61d531f36b72 None admin] RBAC: Adding query filter params (role.id=069fe3e033664493b3def844e3735bc9, scope.project.id=41f15a3cf54f4397920b0e8f758539ea, user.id=3c115932304948d4968c4f07fae9c96b) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:37.836404 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740a9e32-5ffe-43bd-993a-61d531f36b72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:37.838127 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740a9e32-5ffe-43bd-993a-61d531f36b72 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:37.845970 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 84/176] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 8 00:59:37 2022] GET /identity/v3/role_assignments?user.id=3c115932304948d4968c4f07fae9c96b&scope.project.id=41f15a3cf54f4397920b0e8f758539ea&role.id=069fe3e033664493b3def844e3735bc9 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:37.846989 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740a9e32-5ffe-43bd-993a-61d531f36b72 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:38.007631 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a83215-53bb-4f64-8d34-ce005ba2151a None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:38.029326 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:38.062158 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740a9e32-5ffe-43bd-993a-61d531f36b72 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:38.080574 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cf9687-665a-48be-8392-6561a796be95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:38.081484 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf9687-665a-48be-8392-6561a796be95 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:38.081910 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf9687-665a-48be-8392-6561a796be95 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:38.082322 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf9687-665a-48be-8392-6561a796be95 None admin] PATH_INFO: `/v3/users/6839266d85444d5da67f883a0eba81f8` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:38.086073 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cf9687-665a-48be-8392-6561a796be95 None admin] RBAC: Authorizing `identity:get_user(user_id=6839266d85444d5da67f883a0eba81f8)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:38.087699 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cf9687-665a-48be-8392-6561a796be95 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:38.091076 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 93/177] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 8 00:59:37 2022] GET /identity/v3/users/6839266d85444d5da67f883a0eba81f8 => generated 289 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:38.102505 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cf9687-665a-48be-8392-6561a796be95 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:38.150073 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e94f70-9234-4415-8227-a6dd119cff9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:38.150924 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e94f70-9234-4415-8227-a6dd119cff9f None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:38.151365 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e94f70-9234-4415-8227-a6dd119cff9f None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:38.151770 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e94f70-9234-4415-8227-a6dd119cff9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:38.152775 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e94f70-9234-4415-8227-a6dd119cff9f None admin] RBAC: Adding query filter params (role.id=92aa6d6824404d10b4a3be8b4a6bc317, scope.project.id=34348313aa84446d8c5844c3ed29655b, user.id=3c115932304948d4968c4f07fae9c96b) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:38.153241 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e94f70-9234-4415-8227-a6dd119cff9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:38.155950 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e94f70-9234-4415-8227-a6dd119cff9f None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:38.162945 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 85/178] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 8 00:59:37 2022] GET /identity/v3/role_assignments?user.id=3c115932304948d4968c4f07fae9c96b&scope.project.id=34348313aa84446d8c5844c3ed29655b&role.id=92aa6d6824404d10b4a3be8b4a6bc317 => generated 608 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:38.300528 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cf9687-665a-48be-8392-6561a796be95 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:38.344412 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fa962e-f2ce-4b65-b662-6f97d75c37eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:38.345063 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa962e-f2ce-4b65-b662-6f97d75c37eb None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:38.345343 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa962e-f2ce-4b65-b662-6f97d75c37eb None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:38.345617 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa962e-f2ce-4b65-b662-6f97d75c37eb None admin] PATH_INFO: `/v3/projects/41f15a3cf54f4397920b0e8f758539ea` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:38.349487 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fa962e-f2ce-4b65-b662-6f97d75c37eb None admin] RBAC: Authorizing `identity:get_project(project_id=41f15a3cf54f4397920b0e8f758539ea)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:38.350984 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fa962e-f2ce-4b65-b662-6f97d75c37eb None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:38.352920 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 94/179] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 00:59:38 2022] GET /identity/v3/projects/41f15a3cf54f4397920b0e8f758539ea => generated 298 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:38.359598 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e94f70-9234-4415-8227-a6dd119cff9f None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:38.386533 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:38.511230 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e94f70-9234-4415-8227-a6dd119cff9f None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:38.547221 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcbd0d1-9e57-4d17-8665-9b235131d5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:38.547892 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcbd0d1-9e57-4d17-8665-9b235131d5d2 None admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:38.548194 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcbd0d1-9e57-4d17-8665-9b235131d5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:38.548471 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcbd0d1-9e57-4d17-8665-9b235131d5d2 None admin] PATH_INFO: `/v3/projects/41f15a3cf54f4397920b0e8f758539ea/users/6839266d85444d5da67f883a0eba81f8/roles/653150c294214a0da3bdd8d00aacd3aa` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:38.550731 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcbd0d1-9e57-4d17-8665-9b235131d5d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=41f15a3cf54f4397920b0e8f758539ea, user_id=6839266d85444d5da67f883a0eba81f8, role_id=653150c294214a0da3bdd8d00aacd3aa)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:38.552188 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcbd0d1-9e57-4d17-8665-9b235131d5d2 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:38.566862 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 86/180] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 8 00:59:38 2022] PUT /identity/v3/projects/41f15a3cf54f4397920b0e8f758539ea/users/6839266d85444d5da67f883a0eba81f8/roles/653150c294214a0da3bdd8d00aacd3aa => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 00:59:39.493348 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae49ed4-d767-4341-a12c-1690253a8be5 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:39.493899 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae49ed4-d767-4341-a12c-1690253a8be5 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:39.493899 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae49ed4-d767-4341-a12c-1690253a8be5 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:39.495235 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 95/181] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 00:59:39.501451 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c28f5f-972d-474a-ad96-2bde9e2aeb94 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:39.501908 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c28f5f-972d-474a-ad96-2bde9e2aeb94 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:39.501990 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c28f5f-972d-474a-ad96-2bde9e2aeb94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:39.522372 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-56c28f5f-972d-474a-ad96-2bde9e2aeb94 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:39.523617 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-56c28f5f-972d-474a-ad96-2bde9e2aeb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:39.545029 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913810e9-7cdc-47f6-8db2-a226c8906631 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:39.545379 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913810e9-7cdc-47f6-8db2-a226c8906631 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:39.545616 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913810e9-7cdc-47f6-8db2-a226c8906631 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:39.546994 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 96/182] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 00:59:39.552205 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dda89d-38b2-4d0d-b0a3-e573e4643753 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:39.552495 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dda89d-38b2-4d0d-b0a3-e573e4643753 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:39.552762 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dda89d-38b2-4d0d-b0a3-e573e4643753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:39.571748 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-f8dda89d-38b2-4d0d-b0a3-e573e4643753 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:39.572965 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-f8dda89d-38b2-4d0d-b0a3-e573e4643753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:39.599112 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 87/183] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 00:59:39.624803 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:39.650702 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 97/184] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 00:59:39.755714 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83376e33-eeea-4203-9900-cef4ee699210 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:39.756041 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83376e33-eeea-4203-9900-cef4ee699210 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:39.756337 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83376e33-eeea-4203-9900-cef4ee699210 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:39.757675 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 88/185] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 00:59:39.763834 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12febfc2-0086-4254-8c68-00a91f2cceaa None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:39.764116 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12febfc2-0086-4254-8c68-00a91f2cceaa None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:39.764417 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12febfc2-0086-4254-8c68-00a91f2cceaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:39.787099 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-12febfc2-0086-4254-8c68-00a91f2cceaa None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:39.788326 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-12febfc2-0086-4254-8c68-00a91f2cceaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:39.858045 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 98/186] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 00:59:39.897720 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83376e33-eeea-4203-9900-cef4ee699210 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:39.901645 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:39.927178 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12febfc2-0086-4254-8c68-00a91f2cceaa None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:40.038389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12febfc2-0086-4254-8c68-00a91f2cceaa None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:40.038718 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83376e33-eeea-4203-9900-cef4ee699210 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:40.052710 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:40.079773 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e658c6-a160-433b-8c64-22090798b731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:40.080889 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e658c6-a160-433b-8c64-22090798b731 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:40.080995 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f681b88f-fb93-4f7f-ad9f-b08f0b267130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:40.081554 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e658c6-a160-433b-8c64-22090798b731 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:40.081679 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681b88f-fb93-4f7f-ad9f-b08f0b267130 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:40.081880 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681b88f-fb93-4f7f-ad9f-b08f0b267130 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:40.082152 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681b88f-fb93-4f7f-ad9f-b08f0b267130 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:40.082509 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e658c6-a160-433b-8c64-22090798b731 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:40.093390 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f681b88f-fb93-4f7f-ad9f-b08f0b267130 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 00:59:40.095045 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e658c6-a160-433b-8c64-22090798b731 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 00:59:40.098167 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-f681b88f-fb93-4f7f-ad9f-b08f0b267130 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 00:59:40.098167 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 00:59:40.098167 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 00:59:40.098167 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 00:59:40.098167 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 00:59:40.098167 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 00:59:40.098167 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 00:59:40.098167 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 00:59:40.098167 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 00:59:40.098167 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 00:59:40.098167 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 00:59:40.098167 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 00:59:40.098167 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 00:59:40.098167 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 00:59:40.098167 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 00:59:40.098167 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 00:59:40.099025 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 00:59:40.099025 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 00:59:40.099025 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 00:59:40.099025 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 00:59:40.099025 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 00:59:40.099025 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 00:59:40.099025 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 00:59:40.099025 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 00:59:40.099025 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 00:59:40.099025 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 00:59:40.099025 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 00:59:40.099025 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 00:59:40.099025 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 00:59:40.099025 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 00:59:40.099025 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 00:59:40.099025 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 00:59:40.099809 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 00:59:40.099809 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 00:59:40.099809 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 00:59:40.099809 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 00:59:40.099809 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 00:59:40.099809 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 00:59:40.099809 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 00:59:40.099809 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 00:59:40.099809 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 00:59:40.099809 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 00:59:40.099809 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 00:59:40.099809 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 00:59:40.099809 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 89/187] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 8 00:59:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 00:59:40.103166 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-17e658c6-a160-433b-8c64-22090798b731 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 00:59:40.103166 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 00:59:40.103166 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 00:59:40.103166 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 00:59:40.103166 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 00:59:40.103166 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 00:59:40.103166 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 00:59:40.103166 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 00:59:40.103166 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 00:59:40.103166 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 00:59:40.103166 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 00:59:40.103166 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 00:59:40.103166 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 00:59:40.103166 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 00:59:40.103166 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 00:59:40.103166 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 00:59:40.104031 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 00:59:40.104031 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 00:59:40.104031 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 00:59:40.104031 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 00:59:40.104031 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 00:59:40.104031 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 00:59:40.104031 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 00:59:40.104031 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 00:59:40.104031 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 00:59:40.104031 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 00:59:40.104031 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 00:59:40.104031 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 00:59:40.104031 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 00:59:40.104031 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 00:59:40.104031 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 00:59:40.104031 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 00:59:40.104880 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 00:59:40.104880 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 00:59:40.104880 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 00:59:40.104880 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 00:59:40.104880 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 00:59:40.104880 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 00:59:40.104880 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 00:59:40.104880 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 00:59:40.104880 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 00:59:40.104880 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 00:59:40.104880 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 00:59:40.104880 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 00:59:40.105471 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f681b88f-fb93-4f7f-ad9f-b08f0b267130 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:40.107739 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 99/188] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 8 00:59:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 00:59:40.113606 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e658c6-a160-433b-8c64-22090798b731 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:40.170236 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:40.211990 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f681b88f-fb93-4f7f-ad9f-b08f0b267130 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:40.228934 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e658c6-a160-433b-8c64-22090798b731 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:40.247312 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7eb47a-ed4b-462b-9c23-276978b0f8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:40.248475 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7eb47a-ed4b-462b-9c23-276978b0f8ad None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:40.248748 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7eb47a-ed4b-462b-9c23-276978b0f8ad None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:40.249019 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7eb47a-ed4b-462b-9c23-276978b0f8ad None admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:40.249683 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7eb47a-ed4b-462b-9c23-276978b0f8ad None admin] RBAC: Adding query filter params (name=admin) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:40.249993 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7eb47a-ed4b-462b-9c23-276978b0f8ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:40.251380 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7eb47a-ed4b-462b-9c23-276978b0f8ad None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:40.259942 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 90/189] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 00:59:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 00:59:40.261155 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7eb47a-ed4b-462b-9c23-276978b0f8ad None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:40.269564 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11504e14-bc11-42e5-bf60-c798677eca8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:40.270186 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11504e14-bc11-42e5-bf60-c798677eca8f None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:40.270463 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11504e14-bc11-42e5-bf60-c798677eca8f None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:40.270735 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11504e14-bc11-42e5-bf60-c798677eca8f None admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:40.271407 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11504e14-bc11-42e5-bf60-c798677eca8f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:40.271710 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11504e14-bc11-42e5-bf60-c798677eca8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:40.273132 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11504e14-bc11-42e5-bf60-c798677eca8f None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:40.281879 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 100/190] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 00:59:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 00:59:40.282511 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11504e14-bc11-42e5-bf60-c798677eca8f None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:40.317007 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:40.369691 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7eb47a-ed4b-462b-9c23-276978b0f8ad None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:40.397417 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:40.397708 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11504e14-bc11-42e5-bf60-c798677eca8f None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:40.408818 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c07bf28-6b77-4ccb-a0fa-1da1464f0dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:40.409575 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c07bf28-6b77-4ccb-a0fa-1da1464f0dcb None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:40.409859 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c07bf28-6b77-4ccb-a0fa-1da1464f0dcb None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:40.410136 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c07bf28-6b77-4ccb-a0fa-1da1464f0dcb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:40.411254 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c07bf28-6b77-4ccb-a0fa-1da1464f0dcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:40.412879 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c07bf28-6b77-4ccb-a0fa-1da1464f0dcb None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:40.414484 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 91/191] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 00:59:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:40.416959 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c07bf28-6b77-4ccb-a0fa-1da1464f0dcb None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:40.436919 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113c84ef-26ad-4470-a4b6-6450aa23e95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:40.437557 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113c84ef-26ad-4470-a4b6-6450aa23e95c None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:40.437950 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113c84ef-26ad-4470-a4b6-6450aa23e95c None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:40.438223 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113c84ef-26ad-4470-a4b6-6450aa23e95c None admin] PATH_INFO: `/v3/users/a60b663812334e4a83e64d35179025f6` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:40.454460 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113c84ef-26ad-4470-a4b6-6450aa23e95c None admin] RBAC: Authorizing `identity:get_user(user_id=a60b663812334e4a83e64d35179025f6)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:40.455939 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113c84ef-26ad-4470-a4b6-6450aa23e95c None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:40.457787 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 101/192] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 8 00:59:40 2022] GET /identity/v3/users/a60b663812334e4a83e64d35179025f6 => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:40.458638 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113c84ef-26ad-4470-a4b6-6450aa23e95c None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:40.527129 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c07bf28-6b77-4ccb-a0fa-1da1464f0dcb None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:40.558287 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:40.564196 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d2e81c-0f64-41cb-8848-4d29b865bee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:40.564889 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d2e81c-0f64-41cb-8848-4d29b865bee3 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:40.565173 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d2e81c-0f64-41cb-8848-4d29b865bee3 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:40.565494 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d2e81c-0f64-41cb-8848-4d29b865bee3 None admin] PATH_INFO: `/v3/users/a60b663812334e4a83e64d35179025f6` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:40.566705 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d2e81c-0f64-41cb-8848-4d29b865bee3 None admin] RBAC: Authorizing `identity:get_user(user_id=a60b663812334e4a83e64d35179025f6)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:40.568253 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d2e81c-0f64-41cb-8848-4d29b865bee3 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:40.570084 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 92/193] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 8 00:59:40 2022] GET /identity/v3/users/a60b663812334e4a83e64d35179025f6 => generated 283 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:40.570855 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d2e81c-0f64-41cb-8848-4d29b865bee3 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:40.572213 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113c84ef-26ad-4470-a4b6-6450aa23e95c None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:40.611673 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e016129-e7db-4b1c-83ec-919871978165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:40.612577 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e016129-e7db-4b1c-83ec-919871978165 None admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:40.612852 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e016129-e7db-4b1c-83ec-919871978165 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:40.613123 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e016129-e7db-4b1c-83ec-919871978165 None admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:40.614003 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e016129-e7db-4b1c-83ec-919871978165 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:40.615232 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e016129-e7db-4b1c-83ec-919871978165 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:40.642958 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 102/194] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 8 00:59:40 2022] POST /identity/v3/users => generated 297 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 00:59:40.643793 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e016129-e7db-4b1c-83ec-919871978165 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:40.656643 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:40.714203 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d2e81c-0f64-41cb-8848-4d29b865bee3 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:40.768924 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf29cdaf-4f40-4dd9-b723-314d85dfe258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:40.769863 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf29cdaf-4f40-4dd9-b723-314d85dfe258 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:40.770670 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf29cdaf-4f40-4dd9-b723-314d85dfe258 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:40.771026 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf29cdaf-4f40-4dd9-b723-314d85dfe258 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:40.780508 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e016129-e7db-4b1c-83ec-919871978165 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:40.826463 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee4f8ba-4540-44c0-a8a6-fd7508e0be2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:40.827340 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee4f8ba-4540-44c0-a8a6-fd7508e0be2a None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:40.827340 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee4f8ba-4540-44c0-a8a6-fd7508e0be2a None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:40.827543 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee4f8ba-4540-44c0-a8a6-fd7508e0be2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:40.845100 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:40.863770 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf29cdaf-4f40-4dd9-b723-314d85dfe258 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:40.865284 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf29cdaf-4f40-4dd9-b723-314d85dfe258 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:40.911898 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee4f8ba-4540-44c0-a8a6-fd7508e0be2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:40.913959 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee4f8ba-4540-44c0-a8a6-fd7508e0be2a None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:40.932291 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:40.990866 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 93/195] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 00:59:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 00:59:40.992190 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1303f2-0183-4e27-8892-46a406f870e7 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:40.992480 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1303f2-0183-4e27-8892-46a406f870e7 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:40.992750 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1303f2-0183-4e27-8892-46a406f870e7 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:40.994250 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 94/196] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 00:59:40.999239 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1303f2-0183-4e27-8892-46a406f870e7 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:41.047072 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 103/197] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 00:59:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 00:59:41.048453 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5b346-fd5c-4ef5-815c-d5ceeec7ab98 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:41.048737 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5b346-fd5c-4ef5-815c-d5ceeec7ab98 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:41.049006 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5b346-fd5c-4ef5-815c-d5ceeec7ab98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:41.070126 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-a3c5b346-fd5c-4ef5-815c-d5ceeec7ab98 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:41.071307 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-a3c5b346-fd5c-4ef5-815c-d5ceeec7ab98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:41.120107 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:41.128093 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1303f2-0183-4e27-8892-46a406f870e7 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:41.151221 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 104/198] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 00:59:41.152515 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c5b346-fd5c-4ef5-815c-d5ceeec7ab98 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:41.167252 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b4ec7-1e27-48b9-ad6c-441b1c30766c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:41.167986 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b4ec7-1e27-48b9-ad6c-441b1c30766c None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:41.168298 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b4ec7-1e27-48b9-ad6c-441b1c30766c None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:41.168569 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b4ec7-1e27-48b9-ad6c-441b1c30766c None admin] PATH_INFO: `/v3/projects/0f19f058a73f49819b30c33d7fcfc52f` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:41.169801 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361b4ec7-1e27-48b9-ad6c-441b1c30766c None admin] RBAC: Authorizing `identity:get_project(project_id=0f19f058a73f49819b30c33d7fcfc52f)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:41.171824 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361b4ec7-1e27-48b9-ad6c-441b1c30766c None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:41.173862 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 95/199] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 00:59:40 2022] GET /identity/v3/projects/0f19f058a73f49819b30c33d7fcfc52f => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:41.179579 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b4ec7-1e27-48b9-ad6c-441b1c30766c None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:41.214501 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:41.275937 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c5b346-fd5c-4ef5-815c-d5ceeec7ab98 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:41.298348 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b4ec7-1e27-48b9-ad6c-441b1c30766c None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:41.320189 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fb1445-952e-4257-ad10-d2471994f26f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:41.322361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb1445-952e-4257-ad10-d2471994f26f None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:41.322642 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb1445-952e-4257-ad10-d2471994f26f None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:41.322917 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb1445-952e-4257-ad10-d2471994f26f None admin] PATH_INFO: `/v3/projects/0f19f058a73f49819b30c33d7fcfc52f` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:41.330142 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fb1445-952e-4257-ad10-d2471994f26f None admin] RBAC: Authorizing `identity:get_project(project_id=0f19f058a73f49819b30c33d7fcfc52f)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:41.331623 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fb1445-952e-4257-ad10-d2471994f26f None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:41.333361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 105/200] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 00:59:41 2022] GET /identity/v3/projects/0f19f058a73f49819b30c33d7fcfc52f => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:41.339180 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fb1445-952e-4257-ad10-d2471994f26f None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:41.364019 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aeb732-0843-44bf-81f8-316634d9a740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:41.364658 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aeb732-0843-44bf-81f8-316634d9a740 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:41.364937 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aeb732-0843-44bf-81f8-316634d9a740 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:41.365205 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aeb732-0843-44bf-81f8-316634d9a740 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:41.365937 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aeb732-0843-44bf-81f8-316634d9a740 None admin] RBAC: Adding query filter params (role.id=34de369ec7c54715aa83a9052a26c3f2, scope.project.id=0f19f058a73f49819b30c33d7fcfc52f, user.id=a60b663812334e4a83e64d35179025f6) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:41.366240 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aeb732-0843-44bf-81f8-316634d9a740 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:41.367695 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aeb732-0843-44bf-81f8-316634d9a740 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:41.376704 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 96/201] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 8 00:59:41 2022] GET /identity/v3/role_assignments?user.id=a60b663812334e4a83e64d35179025f6&scope.project.id=0f19f058a73f49819b30c33d7fcfc52f&role.id=34de369ec7c54715aa83a9052a26c3f2 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 00:59:41.407803 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:41.477085 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fb1445-952e-4257-ad10-d2471994f26f None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:41.517738 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d17941e-7546-483c-ae29-3bfe08b26126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:41.518362 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d17941e-7546-483c-ae29-3bfe08b26126 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:41.518643 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d17941e-7546-483c-ae29-3bfe08b26126 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:41.518914 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d17941e-7546-483c-ae29-3bfe08b26126 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:41.519673 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d17941e-7546-483c-ae29-3bfe08b26126 None admin] RBAC: Adding query filter params (role.id=069fe3e033664493b3def844e3735bc9, scope.project.id=0f19f058a73f49819b30c33d7fcfc52f, user.id=a60b663812334e4a83e64d35179025f6) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:41.519982 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d17941e-7546-483c-ae29-3bfe08b26126 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:41.521573 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d17941e-7546-483c-ae29-3bfe08b26126 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:41.528395 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 106/202] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 8 00:59:41 2022] GET /identity/v3/role_assignments?user.id=a60b663812334e4a83e64d35179025f6&scope.project.id=0f19f058a73f49819b30c33d7fcfc52f&role.id=069fe3e033664493b3def844e3735bc9 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 00:59:41.587933 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aeb732-0843-44bf-81f8-316634d9a740 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:41.700257 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:41.707538 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aeb732-0843-44bf-81f8-316634d9a740 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:41.753731 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c005e3e1-2e4b-438d-94a5-974067bc97fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:41.754713 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005e3e1-2e4b-438d-94a5-974067bc97fa None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:41.755140 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005e3e1-2e4b-438d-94a5-974067bc97fa None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:41.755550 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005e3e1-2e4b-438d-94a5-974067bc97fa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:41.771197 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c005e3e1-2e4b-438d-94a5-974067bc97fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 00:59:41.776761 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-c005e3e1-2e4b-438d-94a5-974067bc97fa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 00:59:41.776761 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 00:59:41.776761 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 00:59:41.776761 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 00:59:41.776761 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 00:59:41.776761 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 00:59:41.776761 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 00:59:41.776761 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 00:59:41.776761 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 00:59:41.776761 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 00:59:41.776761 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 00:59:41.776761 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 00:59:41.776761 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 00:59:41.776761 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 00:59:41.776761 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 00:59:41.776761 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 00:59:41.777792 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 00:59:41.777792 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 00:59:41.777792 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 00:59:41.777792 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 00:59:41.777792 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 00:59:41.777792 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 00:59:41.777792 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 00:59:41.777792 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 00:59:41.777792 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 00:59:41.777792 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 00:59:41.777792 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 00:59:41.777792 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 00:59:41.777792 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 00:59:41.777792 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 00:59:41.777792 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 00:59:41.777792 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 00:59:41.778761 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 00:59:41.778761 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 00:59:41.778761 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 00:59:41.778761 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 00:59:41.778761 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 00:59:41.778761 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 00:59:41.778761 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 00:59:41.778761 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 00:59:41.778761 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 00:59:41.778761 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 00:59:41.778761 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 00:59:41.778761 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 00:59:41.781182 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 97/203] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 00:59:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 00:59:41.787175 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d17941e-7546-483c-ae29-3bfe08b26126 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:41.921529 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d17941e-7546-483c-ae29-3bfe08b26126 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:41.930686 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:41.957229 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c5775-6037-4f6c-b37e-30d35e811d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:41.957842 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c5775-6037-4f6c-b37e-30d35e811d7f None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:41.958122 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c5775-6037-4f6c-b37e-30d35e811d7f None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:41.958398 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c5775-6037-4f6c-b37e-30d35e811d7f None admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:41.959127 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c5775-6037-4f6c-b37e-30d35e811d7f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:41.959431 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c5775-6037-4f6c-b37e-30d35e811d7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:41.960885 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c5775-6037-4f6c-b37e-30d35e811d7f None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:41.970199 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 107/204] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 8 00:59:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:41.976200 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c005e3e1-2e4b-438d-94a5-974067bc97fa None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:42.084248 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c005e3e1-2e4b-438d-94a5-974067bc97fa None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:42.119637 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08a382c-4c31-457f-869a-8b04093b59fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:42.120302 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a382c-4c31-457f-869a-8b04093b59fc None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:42.120638 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a382c-4c31-457f-869a-8b04093b59fc None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:42.120913 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a382c-4c31-457f-869a-8b04093b59fc None admin] PATH_INFO: `/v3/users/6839266d85444d5da67f883a0eba81f8` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:42.122087 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08a382c-4c31-457f-869a-8b04093b59fc None admin] RBAC: Authorizing `identity:get_user(user_id=6839266d85444d5da67f883a0eba81f8)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:42.123726 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08a382c-4c31-457f-869a-8b04093b59fc None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:42.125399 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 98/205] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 8 00:59:41 2022] GET /identity/v3/users/6839266d85444d5da67f883a0eba81f8 => generated 289 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:42.131129 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c5775-6037-4f6c-b37e-30d35e811d7f None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:42.249948 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c5775-6037-4f6c-b37e-30d35e811d7f None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:42.286418 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6f45c5-9edb-4460-b0bd-2db569601af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:42.287027 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6f45c5-9edb-4460-b0bd-2db569601af6 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:42.287304 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6f45c5-9edb-4460-b0bd-2db569601af6 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:42.287580 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6f45c5-9edb-4460-b0bd-2db569601af6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:42.329099 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:42.359339 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6f45c5-9edb-4460-b0bd-2db569601af6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:42.360895 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6f45c5-9edb-4460-b0bd-2db569601af6 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:42.463984 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 108/206] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 00:59:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 00:59:42.470330 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08a382c-4c31-457f-869a-8b04093b59fc None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:42.496067 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:42.581699 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08a382c-4c31-457f-869a-8b04093b59fc None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:42.633626 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922b27a7-cff6-4fac-a1f9-5719ba2b6381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:42.634272 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b27a7-cff6-4fac-a1f9-5719ba2b6381 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:42.634556 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b27a7-cff6-4fac-a1f9-5719ba2b6381 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:42.634830 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b27a7-cff6-4fac-a1f9-5719ba2b6381 None admin] PATH_INFO: `/v3/projects/41f15a3cf54f4397920b0e8f758539ea` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:42.640837 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922b27a7-cff6-4fac-a1f9-5719ba2b6381 None admin] RBAC: Authorizing `identity:get_project(project_id=41f15a3cf54f4397920b0e8f758539ea)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:42.642311 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922b27a7-cff6-4fac-a1f9-5719ba2b6381 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:42.644041 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 99/207] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 00:59:42 2022] GET /identity/v3/projects/41f15a3cf54f4397920b0e8f758539ea => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:42.650150 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6f45c5-9edb-4460-b0bd-2db569601af6 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:42.765903 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:42.808287 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6f45c5-9edb-4460-b0bd-2db569601af6 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:42.856436 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7900e93-96af-4bf4-ba88-0d08e74a90f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:42.857995 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7900e93-96af-4bf4-ba88-0d08e74a90f2 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:42.858398 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7900e93-96af-4bf4-ba88-0d08e74a90f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:42.858688 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7900e93-96af-4bf4-ba88-0d08e74a90f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:42.859436 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7900e93-96af-4bf4-ba88-0d08e74a90f2 None admin] RBAC: Adding query filter params (role.id=653150c294214a0da3bdd8d00aacd3aa, scope.project.id=41f15a3cf54f4397920b0e8f758539ea, user.id=6839266d85444d5da67f883a0eba81f8) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:42.859748 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7900e93-96af-4bf4-ba88-0d08e74a90f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:42.861261 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7900e93-96af-4bf4-ba88-0d08e74a90f2 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:42.868369 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 109/208] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 8 00:59:42 2022] GET /identity/v3/role_assignments?user.id=6839266d85444d5da67f883a0eba81f8&scope.project.id=41f15a3cf54f4397920b0e8f758539ea&role.id=653150c294214a0da3bdd8d00aacd3aa => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:43.410301 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb84ff-4690-4b94-88c7-35ca76c1049e None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:43.410641 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb84ff-4690-4b94-88c7-35ca76c1049e None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:43.410852 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb84ff-4690-4b94-88c7-35ca76c1049e None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:43.412287 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 100/209] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 00:59:43.418367 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a338059-b8f5-4f59-8d27-ce112c89c941 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:43.418656 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a338059-b8f5-4f59-8d27-ce112c89c941 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:43.418921 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a338059-b8f5-4f59-8d27-ce112c89c941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:43.445926 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-0a338059-b8f5-4f59-8d27-ce112c89c941 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:43.451317 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-0a338059-b8f5-4f59-8d27-ce112c89c941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:43.534082 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 110/210] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 00:59:43.542868 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a2094-7b42-4740-b010-d5d05ef978a9 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:43.543151 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a2094-7b42-4740-b010-d5d05ef978a9 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:43.543423 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a2094-7b42-4740-b010-d5d05ef978a9 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:43.545431 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 101/211] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 00:59:43.550552 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0094a997-9b1d-4db9-a61e-57abb92197f7 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:43.551023 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0094a997-9b1d-4db9-a61e-57abb92197f7 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:43.551430 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0094a997-9b1d-4db9-a61e-57abb92197f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:43.569740 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-0094a997-9b1d-4db9-a61e-57abb92197f7 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:43.571120 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-0094a997-9b1d-4db9-a61e-57abb92197f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:43.636419 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:43.645803 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 111/212] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 00:59:43.802030 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a2094-7b42-4740-b010-d5d05ef978a9 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:43.899845 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:43.915110 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a2094-7b42-4740-b010-d5d05ef978a9 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:43.933601 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0094a997-9b1d-4db9-a61e-57abb92197f7 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:43.952073 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f24a0d8-9a70-4338-a841-501079e49642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:43.952779 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f24a0d8-9a70-4338-a841-501079e49642 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:43.953056 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f24a0d8-9a70-4338-a841-501079e49642 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:43.953330 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f24a0d8-9a70-4338-a841-501079e49642 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:43.964964 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f24a0d8-9a70-4338-a841-501079e49642 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 00:59:43.971168 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-7f24a0d8-9a70-4338-a841-501079e49642 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 00:59:43.971168 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 00:59:43.971168 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 00:59:43.971168 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 00:59:43.971168 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 00:59:43.971168 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 00:59:43.971168 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 00:59:43.971168 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 00:59:43.971168 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 00:59:43.971168 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 00:59:43.971168 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 00:59:43.971168 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 00:59:43.971168 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 00:59:43.971168 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 00:59:43.971168 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 00:59:43.971168 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 00:59:43.972058 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 00:59:43.972058 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 00:59:43.972058 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 00:59:43.972058 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 00:59:43.972058 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 00:59:43.972058 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 00:59:43.972058 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 00:59:43.972058 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 00:59:43.972058 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 00:59:43.972058 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 00:59:43.972058 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 00:59:43.972058 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 00:59:43.972058 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 00:59:43.972058 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 00:59:43.972058 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 00:59:43.972058 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 00:59:43.973209 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 00:59:43.973209 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 00:59:43.973209 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 00:59:43.973209 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 00:59:43.973209 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 00:59:43.973209 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 00:59:43.973209 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 00:59:43.973209 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 00:59:43.973209 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 00:59:43.973209 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 00:59:43.973209 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 00:59:43.973209 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 00:59:43.973209 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 102/213] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 00:59:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 00:59:43.978222 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f24a0d8-9a70-4338-a841-501079e49642 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:44.050977 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0094a997-9b1d-4db9-a61e-57abb92197f7 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:44.089463 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f24a0d8-9a70-4338-a841-501079e49642 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:44.095337 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3825f3d-11b1-43b5-8314-c78a764bb53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:44.095981 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3825f3d-11b1-43b5-8314-c78a764bb53c None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:44.096287 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3825f3d-11b1-43b5-8314-c78a764bb53c None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:44.096559 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3825f3d-11b1-43b5-8314-c78a764bb53c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:44.100088 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3825f3d-11b1-43b5-8314-c78a764bb53c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:44.101731 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3825f3d-11b1-43b5-8314-c78a764bb53c None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:44.104184 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 112/214] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 00:59:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 00:59:44.109733 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3825f3d-11b1-43b5-8314-c78a764bb53c None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:44.144694 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe6b1e9-92bb-465b-b998-aed3251b8032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:44.145892 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe6b1e9-92bb-465b-b998-aed3251b8032 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:44.146787 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe6b1e9-92bb-465b-b998-aed3251b8032 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:44.147585 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe6b1e9-92bb-465b-b998-aed3251b8032 None admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:44.148544 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe6b1e9-92bb-465b-b998-aed3251b8032 None admin] RBAC: Adding query filter params (name=member) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:44.149212 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe6b1e9-92bb-465b-b998-aed3251b8032 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:44.151668 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe6b1e9-92bb-465b-b998-aed3251b8032 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:44.164405 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 103/215] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 8 00:59:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 00:59:44.172186 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe6b1e9-92bb-465b-b998-aed3251b8032 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:44.194378 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:44.197121 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:44.238564 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3825f3d-11b1-43b5-8314-c78a764bb53c None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:44.279635 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7c7bdc-848d-4895-a69b-51c187adba0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:44.280432 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c7bdc-848d-4895-a69b-51c187adba0b None admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:44.280707 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c7bdc-848d-4895-a69b-51c187adba0b None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:44.280999 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c7bdc-848d-4895-a69b-51c187adba0b None admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:44.281773 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7c7bdc-848d-4895-a69b-51c187adba0b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:44.283222 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7c7bdc-848d-4895-a69b-51c187adba0b None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:44.304811 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe6b1e9-92bb-465b-b998-aed3251b8032 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:44.305475 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 113/216] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 8 00:59:44 2022] POST /identity/v3/users => generated 297 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 00:59:44.306677 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67831900-a666-45c4-813f-a58858f3bbe2 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:44.306965 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67831900-a666-45c4-813f-a58858f3bbe2 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:44.307238 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67831900-a666-45c4-813f-a58858f3bbe2 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:44.308507 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 114/217] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 00:59:44.313493 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69634d-4b14-425b-88d8-5f68a1991e6d None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:44.313493 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69634d-4b14-425b-88d8-5f68a1991e6d None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:44.313493 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69634d-4b14-425b-88d8-5f68a1991e6d None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:44.316113 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 115/218] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 00:59:44.317433 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb067e2-a32e-4302-a871-0c15e643d7d8 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:44.317719 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb067e2-a32e-4302-a871-0c15e643d7d8 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:44.317998 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb067e2-a32e-4302-a871-0c15e643d7d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:44.339341 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-2cb067e2-a32e-4302-a871-0c15e643d7d8 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:44.340687 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-2cb067e2-a32e-4302-a871-0c15e643d7d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:44.346187 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957daef6-5565-4923-b799-b67b3418b245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:44.346894 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957daef6-5565-4923-b799-b67b3418b245 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:44.347177 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957daef6-5565-4923-b799-b67b3418b245 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:44.347447 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957daef6-5565-4923-b799-b67b3418b245 None admin] PATH_INFO: `/v3/users/3169b2a903354aa08f4b4415b53d39e2` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:44.363145 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957daef6-5565-4923-b799-b67b3418b245 None admin] RBAC: Authorizing `identity:get_user(user_id=3169b2a903354aa08f4b4415b53d39e2)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:44.364707 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957daef6-5565-4923-b799-b67b3418b245 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:44.366768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 104/219] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 8 00:59:44 2022] GET /identity/v3/users/3169b2a903354aa08f4b4415b53d39e2 => generated 297 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 00:59:44.368021 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e3226d-b1d9-4005-9788-a19e6bb5a3f0 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:44.368341 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e3226d-b1d9-4005-9788-a19e6bb5a3f0 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:44.368614 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e3226d-b1d9-4005-9788-a19e6bb5a3f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:44.389100 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-a3e3226d-b1d9-4005-9788-a19e6bb5a3f0 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:44.390340 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-a3e3226d-b1d9-4005-9788-a19e6bb5a3f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:44.421222 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 116/220] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 00:59:44.421222 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb067e2-a32e-4302-a871-0c15e643d7d8 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:44.485648 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:44.488707 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:44.493476 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 105/221] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 00:59:44.553732 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb067e2-a32e-4302-a871-0c15e643d7d8 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:44.590685 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca98706-d4fe-4f7a-bf0a-df330fb9e5c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:44.591910 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca98706-d4fe-4f7a-bf0a-df330fb9e5c7 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:44.592230 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca98706-d4fe-4f7a-bf0a-df330fb9e5c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:44.592483 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca98706-d4fe-4f7a-bf0a-df330fb9e5c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:44.667374 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca98706-d4fe-4f7a-bf0a-df330fb9e5c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:44.668884 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca98706-d4fe-4f7a-bf0a-df330fb9e5c7 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:44.707081 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e3226d-b1d9-4005-9788-a19e6bb5a3f0 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:44.724813 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:44.793862 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 117/222] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 00:59:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 00:59:44.795277 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca98706-d4fe-4f7a-bf0a-df330fb9e5c7 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:44.838862 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e3226d-b1d9-4005-9788-a19e6bb5a3f0 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:44.878220 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e49e4b3-a561-43cd-aea2-6afc7dbb6c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:44.878789 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e49e4b3-a561-43cd-aea2-6afc7dbb6c02 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:44.879067 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e49e4b3-a561-43cd-aea2-6afc7dbb6c02 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:44.879342 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e49e4b3-a561-43cd-aea2-6afc7dbb6c02 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:44.891199 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e49e4b3-a561-43cd-aea2-6afc7dbb6c02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 00:59:44.896229 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-0e49e4b3-a561-43cd-aea2-6afc7dbb6c02 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 00:59:44.896229 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 00:59:44.896229 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 00:59:44.896229 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 00:59:44.896229 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 00:59:44.896229 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 00:59:44.896229 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 00:59:44.896229 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 00:59:44.896229 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 00:59:44.896229 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 00:59:44.896229 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 00:59:44.896229 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 00:59:44.896229 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 00:59:44.896229 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 00:59:44.896229 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 00:59:44.896229 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 00:59:44.897085 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 00:59:44.897085 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 00:59:44.897085 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 00:59:44.897085 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 00:59:44.897085 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 00:59:44.897085 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 00:59:44.897085 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 00:59:44.897085 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 00:59:44.897085 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 00:59:44.897085 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 00:59:44.897085 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 00:59:44.897085 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 00:59:44.897085 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 00:59:44.897085 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 00:59:44.897085 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 00:59:44.897085 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 00:59:44.897851 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 00:59:44.897851 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 00:59:44.897851 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 00:59:44.897851 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 00:59:44.897851 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 00:59:44.897851 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 00:59:44.897851 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 00:59:44.897851 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 00:59:44.897851 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 00:59:44.897851 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 00:59:44.897851 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 00:59:44.897851 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 00:59:44.897851 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 106/223] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 8 00:59:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 00:59:44.898490 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e49e4b3-a561-43cd-aea2-6afc7dbb6c02 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:44.924755 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca98706-d4fe-4f7a-bf0a-df330fb9e5c7 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:44.967790 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7384ef7-fc68-4f13-92f3-54dd2659b48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:44.968484 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7384ef7-fc68-4f13-92f3-54dd2659b48e None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:44.968771 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7384ef7-fc68-4f13-92f3-54dd2659b48e None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:44.969054 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7384ef7-fc68-4f13-92f3-54dd2659b48e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:44.972768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:44.982753 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7384ef7-fc68-4f13-92f3-54dd2659b48e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 00:59:44.987916 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-c7384ef7-fc68-4f13-92f3-54dd2659b48e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 00:59:44.987916 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 00:59:44.987916 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 00:59:44.987916 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 00:59:44.987916 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 00:59:44.987916 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 00:59:44.987916 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 00:59:44.987916 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 00:59:44.987916 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 00:59:44.987916 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 00:59:44.987916 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 00:59:44.987916 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 00:59:44.987916 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 00:59:44.987916 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 00:59:44.987916 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 00:59:44.987916 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 00:59:44.988845 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 00:59:44.988845 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 00:59:44.988845 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 00:59:44.988845 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 00:59:44.988845 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 00:59:44.988845 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 00:59:44.988845 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 00:59:44.988845 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 00:59:44.988845 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 00:59:44.988845 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 00:59:44.988845 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 00:59:44.988845 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 00:59:44.988845 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 00:59:44.988845 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 00:59:44.988845 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 00:59:44.988845 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 00:59:44.989637 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 00:59:44.989637 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 00:59:44.989637 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 00:59:44.989637 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 00:59:44.989637 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 00:59:44.989637 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 00:59:44.989637 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 00:59:44.989637 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 00:59:44.989637 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 00:59:44.989637 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 00:59:44.989637 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 00:59:44.989637 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 00:59:44.989637 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 118/224] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 8 00:59:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 00:59:44.990275 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7384ef7-fc68-4f13-92f3-54dd2659b48e None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:45.001947 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:45.022108 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e49e4b3-a561-43cd-aea2-6afc7dbb6c02 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:45.058854 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e1fda-4802-4a62-96b8-96dc35dfa003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:45.059732 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e1fda-4802-4a62-96b8-96dc35dfa003 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:45.060450 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e1fda-4802-4a62-96b8-96dc35dfa003 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:45.061072 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e1fda-4802-4a62-96b8-96dc35dfa003 None admin] PATH_INFO: `/v3/projects/0f19f058a73f49819b30c33d7fcfc52f` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:45.062409 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161e1fda-4802-4a62-96b8-96dc35dfa003 None admin] RBAC: Authorizing `identity:get_project(project_id=0f19f058a73f49819b30c33d7fcfc52f)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:45.064020 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161e1fda-4802-4a62-96b8-96dc35dfa003 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:45.065931 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 107/225] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 00:59:44 2022] GET /identity/v3/projects/0f19f058a73f49819b30c33d7fcfc52f => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:45.068203 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e1fda-4802-4a62-96b8-96dc35dfa003 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:45.100517 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7384ef7-fc68-4f13-92f3-54dd2659b48e None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:45.140119 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf1db4-49fa-4f8b-b2aa-ced58034d70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:45.141325 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf1db4-49fa-4f8b-b2aa-ced58034d70e None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:45.141752 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf1db4-49fa-4f8b-b2aa-ced58034d70e None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:45.142366 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf1db4-49fa-4f8b-b2aa-ced58034d70e None admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:45.143289 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bf1db4-49fa-4f8b-b2aa-ced58034d70e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:45.143738 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bf1db4-49fa-4f8b-b2aa-ced58034d70e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:45.145481 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bf1db4-49fa-4f8b-b2aa-ced58034d70e None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:45.155767 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 119/226] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 00:59:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:45.161122 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf1db4-49fa-4f8b-b2aa-ced58034d70e None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:45.178669 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e1fda-4802-4a62-96b8-96dc35dfa003 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:45.213625 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:45.224367 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b68948f-af2c-4aeb-a925-2dd5dee52ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:45.225100 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b68948f-af2c-4aeb-a925-2dd5dee52ba2 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:45.225384 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b68948f-af2c-4aeb-a925-2dd5dee52ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:45.225666 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b68948f-af2c-4aeb-a925-2dd5dee52ba2 None admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:45.226880 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b68948f-af2c-4aeb-a925-2dd5dee52ba2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:45.227205 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b68948f-af2c-4aeb-a925-2dd5dee52ba2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:45.228776 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b68948f-af2c-4aeb-a925-2dd5dee52ba2 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:45.238810 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 108/227] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 00:59:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:45.239633 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b68948f-af2c-4aeb-a925-2dd5dee52ba2 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:45.260020 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:45.284956 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf1db4-49fa-4f8b-b2aa-ced58034d70e None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:45.322541 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830cf4b8-1df3-4ef5-afa2-120704a012cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:45.323432 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830cf4b8-1df3-4ef5-afa2-120704a012cf None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:45.323719 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830cf4b8-1df3-4ef5-afa2-120704a012cf None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:45.323993 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830cf4b8-1df3-4ef5-afa2-120704a012cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:45.324775 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830cf4b8-1df3-4ef5-afa2-120704a012cf None admin] RBAC: Adding query filter params (role.id=92aa6d6824404d10b4a3be8b4a6bc317, scope.project.id=0f19f058a73f49819b30c33d7fcfc52f, user.id=3169b2a903354aa08f4b4415b53d39e2) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:45.325088 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830cf4b8-1df3-4ef5-afa2-120704a012cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:45.327184 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830cf4b8-1df3-4ef5-afa2-120704a012cf None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:45.333651 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 120/228] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 8 00:59:45 2022] GET /identity/v3/role_assignments?user.id=3169b2a903354aa08f4b4415b53d39e2&scope.project.id=0f19f058a73f49819b30c33d7fcfc52f&role.id=92aa6d6824404d10b4a3be8b4a6bc317 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:45.334455 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830cf4b8-1df3-4ef5-afa2-120704a012cf None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:45.377593 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b68948f-af2c-4aeb-a925-2dd5dee52ba2 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:45.418175 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a84c328-d5e1-42f3-85c2-d5a73b76d45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:45.419155 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84c328-d5e1-42f3-85c2-d5a73b76d45f None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:45.419696 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84c328-d5e1-42f3-85c2-d5a73b76d45f None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:45.420199 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84c328-d5e1-42f3-85c2-d5a73b76d45f None admin] PATH_INFO: `/v3/users/a60b663812334e4a83e64d35179025f6` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:45.421825 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a84c328-d5e1-42f3-85c2-d5a73b76d45f None admin] RBAC: Authorizing `identity:get_user(user_id=a60b663812334e4a83e64d35179025f6)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:45.423594 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a84c328-d5e1-42f3-85c2-d5a73b76d45f None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:45.425498 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 109/229] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 8 00:59:45 2022] GET /identity/v3/users/a60b663812334e4a83e64d35179025f6 => generated 283 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:45.442056 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a84c328-d5e1-42f3-85c2-d5a73b76d45f None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:45.450106 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830cf4b8-1df3-4ef5-afa2-120704a012cf None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:45.489736 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747a0bc9-d8cb-491f-bfd3-65f8eeee3f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:45.490459 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a0bc9-d8cb-491f-bfd3-65f8eeee3f5c None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:45.490664 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a0bc9-d8cb-491f-bfd3-65f8eeee3f5c None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:45.490947 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a0bc9-d8cb-491f-bfd3-65f8eeee3f5c None admin] PATH_INFO: `/v3/users/a60b663812334e4a83e64d35179025f6` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:45.494449 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747a0bc9-d8cb-491f-bfd3-65f8eeee3f5c None admin] RBAC: Authorizing `identity:get_user(user_id=a60b663812334e4a83e64d35179025f6)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:45.494449 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747a0bc9-d8cb-491f-bfd3-65f8eeee3f5c None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:45.495990 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 121/230] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 8 00:59:45 2022] GET /identity/v3/users/a60b663812334e4a83e64d35179025f6 => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:45.502521 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747a0bc9-d8cb-491f-bfd3-65f8eeee3f5c None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:45.508220 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:45.581252 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:45.642264 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a84c328-d5e1-42f3-85c2-d5a73b76d45f None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:45.681490 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70572da5-f652-4b4a-9478-74a209f70479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:45.682044 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70572da5-f652-4b4a-9478-74a209f70479 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:45.682317 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70572da5-f652-4b4a-9478-74a209f70479 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:45.682587 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70572da5-f652-4b4a-9478-74a209f70479 None admin] PATH_INFO: `/v3/projects/0f19f058a73f49819b30c33d7fcfc52f` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:45.683733 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70572da5-f652-4b4a-9478-74a209f70479 None admin] RBAC: Authorizing `identity:get_project(project_id=0f19f058a73f49819b30c33d7fcfc52f)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:45.686458 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70572da5-f652-4b4a-9478-74a209f70479 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:45.688590 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 110/231] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 00:59:45 2022] GET /identity/v3/projects/0f19f058a73f49819b30c33d7fcfc52f => generated 302 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 00:59:45.694994 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70572da5-f652-4b4a-9478-74a209f70479 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:45.769559 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747a0bc9-d8cb-491f-bfd3-65f8eeee3f5c None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:45.831050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87b1109-18fe-4ab2-8f4b-54a6907d266c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:45.832205 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b1109-18fe-4ab2-8f4b-54a6907d266c None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:45.832711 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b1109-18fe-4ab2-8f4b-54a6907d266c None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:45.833467 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b1109-18fe-4ab2-8f4b-54a6907d266c None admin] PATH_INFO: `/v3/projects/0f19f058a73f49819b30c33d7fcfc52f` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:45.834801 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87b1109-18fe-4ab2-8f4b-54a6907d266c None admin] RBAC: Authorizing `identity:get_project(project_id=0f19f058a73f49819b30c33d7fcfc52f)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:45.836944 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87b1109-18fe-4ab2-8f4b-54a6907d266c None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:45.838891 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 122/232] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 00:59:45 2022] GET /identity/v3/projects/0f19f058a73f49819b30c33d7fcfc52f => generated 302 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:45.845385 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87b1109-18fe-4ab2-8f4b-54a6907d266c None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:45.864099 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70572da5-f652-4b4a-9478-74a209f70479 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:45.928325 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:45.944902 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2a5f74-9743-4869-bb16-dc0441649656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:45.945497 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a5f74-9743-4869-bb16-dc0441649656 None admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:45.945773 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a5f74-9743-4869-bb16-dc0441649656 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:45.946048 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a5f74-9743-4869-bb16-dc0441649656 None admin] PATH_INFO: `/v3/projects/0f19f058a73f49819b30c33d7fcfc52f/users/a60b663812334e4a83e64d35179025f6/roles/069fe3e033664493b3def844e3735bc9` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:45.950443 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2a5f74-9743-4869-bb16-dc0441649656 None admin] RBAC: Authorizing `identity:create_grant(project_id=0f19f058a73f49819b30c33d7fcfc52f, user_id=a60b663812334e4a83e64d35179025f6, role_id=069fe3e033664493b3def844e3735bc9)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:45.952067 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2a5f74-9743-4869-bb16-dc0441649656 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:45.983442 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 111/233] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 8 00:59:45 2022] PUT /identity/v3/projects/0f19f058a73f49819b30c33d7fcfc52f/users/a60b663812334e4a83e64d35179025f6/roles/069fe3e033664493b3def844e3735bc9 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 00:59:46.004533 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:46.014158 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87b1109-18fe-4ab2-8f4b-54a6907d266c None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:46.050789 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0c4261-5328-44f1-8c1f-5b036bb3bac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:46.051458 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c4261-5328-44f1-8c1f-5b036bb3bac0 None admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:46.051735 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c4261-5328-44f1-8c1f-5b036bb3bac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:46.052004 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c4261-5328-44f1-8c1f-5b036bb3bac0 None admin] PATH_INFO: `/v3/projects/0f19f058a73f49819b30c33d7fcfc52f/users/a60b663812334e4a83e64d35179025f6/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:46.054011 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0c4261-5328-44f1-8c1f-5b036bb3bac0 None admin] RBAC: Authorizing `identity:create_grant(project_id=0f19f058a73f49819b30c33d7fcfc52f, user_id=a60b663812334e4a83e64d35179025f6, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:46.055495 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0c4261-5328-44f1-8c1f-5b036bb3bac0 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:46.070760 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 123/234] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 8 00:59:45 2022] PUT /identity/v3/projects/0f19f058a73f49819b30c33d7fcfc52f/users/a60b663812334e4a83e64d35179025f6/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 00:59:46.915436 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe9da5-83d6-4013-8495-855530c4e606 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:46.915936 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe9da5-83d6-4013-8495-855530c4e606 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:46.915936 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe9da5-83d6-4013-8495-855530c4e606 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:46.917294 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 112/235] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 00:59:46.922843 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a135a205-d8a8-400d-ae63-0310509861df None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:46.923134 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a135a205-d8a8-400d-ae63-0310509861df None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:46.923417 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a135a205-d8a8-400d-ae63-0310509861df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:46.953153 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-a135a205-d8a8-400d-ae63-0310509861df None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:46.954370 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-a135a205-d8a8-400d-ae63-0310509861df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:47.048614 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 124/236] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 00:59:47.363673 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fe9da5-83d6-4013-8495-855530c4e606 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:47.494026 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fe9da5-83d6-4013-8495-855530c4e606 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:47.504816 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9839539-5cbd-4a40-82f8-a0341e133ade None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:47.505107 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9839539-5cbd-4a40-82f8-a0341e133ade None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:47.505368 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9839539-5cbd-4a40-82f8-a0341e133ade None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:47.507054 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 125/237] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 00:59:47.519388 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f0d33-7454-4188-9bce-9bddafc5df56 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:47.519753 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f0d33-7454-4188-9bce-9bddafc5df56 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:47.520097 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f0d33-7454-4188-9bce-9bddafc5df56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:47.537289 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8204accc-258d-404f-85f0-fc8b5f61f0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:47.538184 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8204accc-258d-404f-85f0-fc8b5f61f0fd None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:47.538664 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8204accc-258d-404f-85f0-fc8b5f61f0fd None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:47.539094 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8204accc-258d-404f-85f0-fc8b5f61f0fd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:47.556189 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-f53f0d33-7454-4188-9bce-9bddafc5df56 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:47.556189 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-f53f0d33-7454-4188-9bce-9bddafc5df56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:47.559484 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8204accc-258d-404f-85f0-fc8b5f61f0fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 00:59:47.565645 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-8204accc-258d-404f-85f0-fc8b5f61f0fd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 00:59:47.565645 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 00:59:47.565645 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 00:59:47.565645 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 00:59:47.565645 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 00:59:47.565645 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 00:59:47.565645 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 00:59:47.565645 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 00:59:47.565645 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 00:59:47.565645 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 00:59:47.565645 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 00:59:47.565645 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 00:59:47.565645 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 00:59:47.565645 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 00:59:47.565645 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 00:59:47.565645 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 00:59:47.566553 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 00:59:47.566553 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 00:59:47.566553 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 00:59:47.566553 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 00:59:47.566553 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 00:59:47.566553 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 00:59:47.566553 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 00:59:47.566553 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 00:59:47.566553 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 00:59:47.566553 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 00:59:47.566553 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 00:59:47.566553 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 00:59:47.566553 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 00:59:47.566553 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 00:59:47.566553 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 00:59:47.566553 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 00:59:47.567333 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 00:59:47.567333 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 00:59:47.567333 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 00:59:47.567333 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 00:59:47.567333 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 00:59:47.567333 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 00:59:47.567333 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 00:59:47.567333 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 00:59:47.567333 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 00:59:47.567333 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 00:59:47.567333 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 00:59:47.567333 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 00:59:47.569217 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 113/238] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 00:59:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 00:59:47.581197 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8204accc-258d-404f-85f0-fc8b5f61f0fd None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:47.658602 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:47.678671 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:47.699927 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 126/239] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 00:59:47.735940 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8204accc-258d-404f-85f0-fc8b5f61f0fd None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:47.778904 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3460b06-63d5-42ec-bf29-7e1317ee5cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:47.779977 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3460b06-63d5-42ec-bf29-7e1317ee5cbd None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:47.780441 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3460b06-63d5-42ec-bf29-7e1317ee5cbd None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:47.780849 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3460b06-63d5-42ec-bf29-7e1317ee5cbd None admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:47.781648 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3460b06-63d5-42ec-bf29-7e1317ee5cbd None admin] RBAC: Adding query filter params (name=reader) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:47.782102 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3460b06-63d5-42ec-bf29-7e1317ee5cbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:47.783682 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3460b06-63d5-42ec-bf29-7e1317ee5cbd None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:47.794936 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 114/240] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 8 00:59:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:47.801095 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53f0d33-7454-4188-9bce-9bddafc5df56 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:47.928389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53f0d33-7454-4188-9bce-9bddafc5df56 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:47.976984 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2796bb74-0b7a-454c-a822-c59b2fa0d83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:47.977851 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2796bb74-0b7a-454c-a822-c59b2fa0d83f None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:47.978295 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2796bb74-0b7a-454c-a822-c59b2fa0d83f None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:47.978704 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2796bb74-0b7a-454c-a822-c59b2fa0d83f None admin] PATH_INFO: `/v3/users/ca8b7995da7e4dfaa839b2c9579a9577` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:47.994770 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6cacaa-6691-414a-9142-43c1bbedbd44 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:47.995254 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6cacaa-6691-414a-9142-43c1bbedbd44 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:47.995357 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6cacaa-6691-414a-9142-43c1bbedbd44 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:47.996840 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 115/241] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 00:59:48.001260 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2796bb74-0b7a-454c-a822-c59b2fa0d83f None admin] RBAC: Authorizing `identity:get_user(user_id=ca8b7995da7e4dfaa839b2c9579a9577)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:48.002788 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2796bb74-0b7a-454c-a822-c59b2fa0d83f None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:48.005820 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e0b09b-adbb-4370-881f-ac4b8c3672a4 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:48.006097 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e0b09b-adbb-4370-881f-ac4b8c3672a4 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:48.006372 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e0b09b-adbb-4370-881f-ac4b8c3672a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:48.008311 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 127/242] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 8 00:59:47 2022] GET /identity/v3/users/ca8b7995da7e4dfaa839b2c9579a9577 => generated 297 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:48.010030 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2796bb74-0b7a-454c-a822-c59b2fa0d83f None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:48.027101 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-83e0b09b-adbb-4370-881f-ac4b8c3672a4 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:48.028321 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-83e0b09b-adbb-4370-881f-ac4b8c3672a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:48.067415 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:48.138326 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2796bb74-0b7a-454c-a822-c59b2fa0d83f None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:48.139960 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 116/243] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 00:59:48.140971 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e0b09b-adbb-4370-881f-ac4b8c3672a4 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:48.179686 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcfb235-8a8f-4286-9b85-800dad22ae4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:48.180946 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcfb235-8a8f-4286-9b85-800dad22ae4f None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:48.181392 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcfb235-8a8f-4286-9b85-800dad22ae4f None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:48.181800 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcfb235-8a8f-4286-9b85-800dad22ae4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:48.237327 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:48.261022 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcfb235-8a8f-4286-9b85-800dad22ae4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:48.262724 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcfb235-8a8f-4286-9b85-800dad22ae4f None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:48.317761 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:48.354593 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e0b09b-adbb-4370-881f-ac4b8c3672a4 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:48.409170 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 128/244] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 00:59:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 08 00:59:48.411444 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcfb235-8a8f-4286-9b85-800dad22ae4f None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:48.414406 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0032f0-a011-4e51-95fa-076d4be38356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:48.415056 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0032f0-a011-4e51-95fa-076d4be38356 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:48.415338 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0032f0-a011-4e51-95fa-076d4be38356 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:48.415613 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0032f0-a011-4e51-95fa-076d4be38356 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:48.417692 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0032f0-a011-4e51-95fa-076d4be38356 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:48.418999 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0032f0-a011-4e51-95fa-076d4be38356 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:48.426055 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 117/245] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 00:59:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:48.426715 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0032f0-a011-4e51-95fa-076d4be38356 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:48.548544 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcfb235-8a8f-4286-9b85-800dad22ae4f None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:48.571349 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0032f0-a011-4e51-95fa-076d4be38356 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:48.581218 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:48.593853 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4a5299-b47f-40b3-a6f3-46540bcb7be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:48.594554 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4a5299-b47f-40b3-a6f3-46540bcb7be3 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:48.594840 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4a5299-b47f-40b3-a6f3-46540bcb7be3 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:48.595508 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4a5299-b47f-40b3-a6f3-46540bcb7be3 None admin] PATH_INFO: `/v3/projects/0f19f058a73f49819b30c33d7fcfc52f` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:48.596688 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4a5299-b47f-40b3-a6f3-46540bcb7be3 None admin] RBAC: Authorizing `identity:get_project(project_id=0f19f058a73f49819b30c33d7fcfc52f)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:48.598255 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4a5299-b47f-40b3-a6f3-46540bcb7be3 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:48.600594 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 129/246] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 00:59:48 2022] GET /identity/v3/projects/0f19f058a73f49819b30c33d7fcfc52f => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:48.601769 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4a5299-b47f-40b3-a6f3-46540bcb7be3 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:48.623987 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e7d7e2-b2a7-4af3-a1d1-4108de498ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:48.624746 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e7d7e2-b2a7-4af3-a1d1-4108de498ccd None admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:48.625032 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e7d7e2-b2a7-4af3-a1d1-4108de498ccd None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:48.625304 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e7d7e2-b2a7-4af3-a1d1-4108de498ccd None admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:48.626002 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e7d7e2-b2a7-4af3-a1d1-4108de498ccd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:48.629299 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:48.634753 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e7d7e2-b2a7-4af3-a1d1-4108de498ccd None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:48.676486 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 118/247] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 8 00:59:48 2022] POST /identity/v3/users => generated 293 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 00:59:48.678161 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e7d7e2-b2a7-4af3-a1d1-4108de498ccd None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:48.775153 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4a5299-b47f-40b3-a6f3-46540bcb7be3 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:48.814024 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee88f36c-226d-4af2-aa5b-56dd7023607a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:48.815263 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee88f36c-226d-4af2-aa5b-56dd7023607a None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:48.815686 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee88f36c-226d-4af2-aa5b-56dd7023607a None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:48.816119 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee88f36c-226d-4af2-aa5b-56dd7023607a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:48.833379 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee88f36c-226d-4af2-aa5b-56dd7023607a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 00:59:48.837012 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e7d7e2-b2a7-4af3-a1d1-4108de498ccd None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:48.838765 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-ee88f36c-226d-4af2-aa5b-56dd7023607a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 00:59:48.838765 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 00:59:48.838765 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 00:59:48.838765 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 00:59:48.838765 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 00:59:48.838765 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 00:59:48.838765 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 00:59:48.838765 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 00:59:48.838765 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 00:59:48.838765 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 00:59:48.838765 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 00:59:48.838765 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 00:59:48.838765 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 00:59:48.838765 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 00:59:48.838765 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 00:59:48.838765 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 00:59:48.840555 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 00:59:48.840555 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 00:59:48.840555 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 00:59:48.840555 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 00:59:48.840555 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 00:59:48.840555 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 00:59:48.840555 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 00:59:48.840555 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 00:59:48.840555 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 00:59:48.840555 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 00:59:48.840555 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 00:59:48.840555 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 00:59:48.840555 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 00:59:48.840555 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 00:59:48.840555 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 00:59:48.840555 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 00:59:48.842389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 00:59:48.842389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 00:59:48.842389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 00:59:48.842389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 00:59:48.842389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 00:59:48.842389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 00:59:48.842389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 00:59:48.842389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 00:59:48.842389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 00:59:48.842389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 00:59:48.842389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 00:59:48.842389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 00:59:48.842389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 130/248] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 00:59:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 00:59:48.849791 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa4ab5-40ec-42b7-8a08-cf6303569b5b None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:48.850176 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa4ab5-40ec-42b7-8a08-cf6303569b5b None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:48.850456 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa4ab5-40ec-42b7-8a08-cf6303569b5b None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:48.852170 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 131/249] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:48 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 00:59:48.852692 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aa4ab5-40ec-42b7-8a08-cf6303569b5b None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:48.886690 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d907f8-7417-4147-815e-51e307c33192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:48.887364 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d907f8-7417-4147-815e-51e307c33192 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:48.887760 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d907f8-7417-4147-815e-51e307c33192 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:48.888160 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d907f8-7417-4147-815e-51e307c33192 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:48.889030 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d907f8-7417-4147-815e-51e307c33192 None admin] RBAC: Adding query filter params (role.id=653150c294214a0da3bdd8d00aacd3aa, scope.project.id=0f19f058a73f49819b30c33d7fcfc52f, user.id=ca8b7995da7e4dfaa839b2c9579a9577) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:48.889346 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d907f8-7417-4147-815e-51e307c33192 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:48.890823 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d907f8-7417-4147-815e-51e307c33192 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:48.898264 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 119/250] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 8 00:59:48 2022] GET /identity/v3/role_assignments?user.id=ca8b7995da7e4dfaa839b2c9579a9577&scope.project.id=0f19f058a73f49819b30c33d7fcfc52f&role.id=653150c294214a0da3bdd8d00aacd3aa => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:48.902691 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a1cec-5d5b-460b-8601-ad1ce8465cb1 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:48.902691 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a1cec-5d5b-460b-8601-ad1ce8465cb1 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:48.902691 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a1cec-5d5b-460b-8601-ad1ce8465cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:48.921371 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-f29a1cec-5d5b-460b-8601-ad1ce8465cb1 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:48.922622 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-f29a1cec-5d5b-460b-8601-ad1ce8465cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:48.948934 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:48.967417 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:49.010125 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aa4ab5-40ec-42b7-8a08-cf6303569b5b None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:49.026686 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 120/251] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 00:59:49.028492 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e5301-ab44-489c-9235-65c619e9e9eb None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:49.028834 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e5301-ab44-489c-9235-65c619e9e9eb None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:49.029201 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e5301-ab44-489c-9235-65c619e9e9eb None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:49.031089 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 121/252] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 00:59:49.037394 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88307ca4-7457-4cb9-a217-9a6692a0b6f0 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:49.037698 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88307ca4-7457-4cb9-a217-9a6692a0b6f0 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:49.037924 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88307ca4-7457-4cb9-a217-9a6692a0b6f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:49.055947 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60665339-3b1d-485d-bc8d-d33bbbb59a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:49.057109 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60665339-3b1d-485d-bc8d-d33bbbb59a7b None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:49.057580 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60665339-3b1d-485d-bc8d-d33bbbb59a7b None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:49.058084 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60665339-3b1d-485d-bc8d-d33bbbb59a7b None admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:49.058945 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60665339-3b1d-485d-bc8d-d33bbbb59a7b None admin] RBAC: Adding query filter params (name=member) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:49.059468 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60665339-3b1d-485d-bc8d-d33bbbb59a7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:49.061951 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-88307ca4-7457-4cb9-a217-9a6692a0b6f0 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:49.062823 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60665339-3b1d-485d-bc8d-d33bbbb59a7b None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:49.063161 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-88307ca4-7457-4cb9-a217-9a6692a0b6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:49.073974 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 132/253] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 8 00:59:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:49.081007 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60665339-3b1d-485d-bc8d-d33bbbb59a7b None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:49.144618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 122/254] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 00:59:49.194552 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60665339-3b1d-485d-bc8d-d33bbbb59a7b None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:49.231557 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe26ae-a137-4093-8a16-8e74ee4050ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:49.232961 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe26ae-a137-4093-8a16-8e74ee4050ff None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:49.233464 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe26ae-a137-4093-8a16-8e74ee4050ff None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:49.233942 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe26ae-a137-4093-8a16-8e74ee4050ff None admin] PATH_INFO: `/v3/users/3169b2a903354aa08f4b4415b53d39e2` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:49.235279 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfe26ae-a137-4093-8a16-8e74ee4050ff None admin] RBAC: Authorizing `identity:get_user(user_id=3169b2a903354aa08f4b4415b53d39e2)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:49.236957 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfe26ae-a137-4093-8a16-8e74ee4050ff None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:49.238878 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 133/255] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 8 00:59:49 2022] GET /identity/v3/users/3169b2a903354aa08f4b4415b53d39e2 => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 00:59:49.244385 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88307ca4-7457-4cb9-a217-9a6692a0b6f0 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:49.352319 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:49.370307 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88307ca4-7457-4cb9-a217-9a6692a0b6f0 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:49.379471 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe26ae-a137-4093-8a16-8e74ee4050ff None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:49.389478 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:49.426911 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e046e2-f761-4ec0-8026-903b05064055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:49.427595 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e046e2-f761-4ec0-8026-903b05064055 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:49.428175 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e046e2-f761-4ec0-8026-903b05064055 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:49.428175 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e046e2-f761-4ec0-8026-903b05064055 None admin] PATH_INFO: `/v3/projects/0f19f058a73f49819b30c33d7fcfc52f` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:49.430772 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e046e2-f761-4ec0-8026-903b05064055 None admin] RBAC: Authorizing `identity:get_project(project_id=0f19f058a73f49819b30c33d7fcfc52f)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:49.432266 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e046e2-f761-4ec0-8026-903b05064055 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:49.434366 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 123/256] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 00:59:49 2022] GET /identity/v3/projects/0f19f058a73f49819b30c33d7fcfc52f => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:49.440096 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e046e2-f761-4ec0-8026-903b05064055 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:49.522332 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe26ae-a137-4093-8a16-8e74ee4050ff None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:49.577433 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e046e2-f761-4ec0-8026-903b05064055 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:49.599263 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45cb953-2920-499d-8511-2f94d1ba4209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:49.599948 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45cb953-2920-499d-8511-2f94d1ba4209 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:49.600247 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45cb953-2920-499d-8511-2f94d1ba4209 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:49.600518 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45cb953-2920-499d-8511-2f94d1ba4209 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:49.612732 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45cb953-2920-499d-8511-2f94d1ba4209 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 00:59:49.620389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-c45cb953-2920-499d-8511-2f94d1ba4209 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 00:59:49.620389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 00:59:49.620389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 00:59:49.620389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 00:59:49.620389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 00:59:49.620389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 00:59:49.620389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 00:59:49.620389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 00:59:49.620389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 00:59:49.620389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 00:59:49.620389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 00:59:49.620389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 00:59:49.620389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 00:59:49.620389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 00:59:49.620389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 00:59:49.620389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 00:59:49.621315 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 00:59:49.621315 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 00:59:49.621315 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 00:59:49.621315 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 00:59:49.621315 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 00:59:49.621315 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 00:59:49.621315 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 00:59:49.621315 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 00:59:49.621315 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 00:59:49.621315 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 00:59:49.621315 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 00:59:49.621315 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 00:59:49.621315 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 00:59:49.621315 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 00:59:49.621315 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 00:59:49.621315 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 00:59:49.622123 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 00:59:49.622123 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 00:59:49.622123 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 00:59:49.622123 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 00:59:49.622123 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 00:59:49.622123 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 00:59:49.622123 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 00:59:49.622123 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 00:59:49.622123 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 00:59:49.622123 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 00:59:49.622123 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 00:59:49.622123 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 00:59:49.622123 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 134/257] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 8 00:59:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 00:59:49.623059 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45cb953-2920-499d-8511-2f94d1ba4209 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:49.652845 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e89a81-50ce-4311-80bd-0355e7d315a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:49.653528 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e89a81-50ce-4311-80bd-0355e7d315a9 None admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:49.654340 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e89a81-50ce-4311-80bd-0355e7d315a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:49.654803 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e89a81-50ce-4311-80bd-0355e7d315a9 None admin] PATH_INFO: `/v3/projects/0f19f058a73f49819b30c33d7fcfc52f/users/3169b2a903354aa08f4b4415b53d39e2/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:49.660897 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e89a81-50ce-4311-80bd-0355e7d315a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=0f19f058a73f49819b30c33d7fcfc52f, user_id=3169b2a903354aa08f4b4415b53d39e2, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:49.662370 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e89a81-50ce-4311-80bd-0355e7d315a9 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:49.711333 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 124/258] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 8 00:59:49 2022] PUT /identity/v3/projects/0f19f058a73f49819b30c33d7fcfc52f/users/3169b2a903354aa08f4b4415b53d39e2/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 00:59:49.712299 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e89a81-50ce-4311-80bd-0355e7d315a9 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:49.717258 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:49.738446 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45cb953-2920-499d-8511-2f94d1ba4209 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:49.769539 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:49.780693 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aaec3e-4cd5-493c-ba9d-57e20fb34956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:49.781471 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aaec3e-4cd5-493c-ba9d-57e20fb34956 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:49.781847 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aaec3e-4cd5-493c-ba9d-57e20fb34956 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:49.782213 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aaec3e-4cd5-493c-ba9d-57e20fb34956 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:49.796325 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aaec3e-4cd5-493c-ba9d-57e20fb34956 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 00:59:49.802380 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-c4aaec3e-4cd5-493c-ba9d-57e20fb34956 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 00:59:49.802380 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 00:59:49.802380 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 00:59:49.802380 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 00:59:49.802380 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 00:59:49.802380 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 00:59:49.802380 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 00:59:49.802380 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 00:59:49.802380 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 00:59:49.802380 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 00:59:49.802380 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 00:59:49.802380 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 00:59:49.802380 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 00:59:49.802380 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 00:59:49.802380 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 00:59:49.802380 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 00:59:49.803371 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 00:59:49.803371 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 00:59:49.803371 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 00:59:49.803371 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 00:59:49.803371 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 00:59:49.803371 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 00:59:49.803371 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 00:59:49.803371 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 00:59:49.803371 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 00:59:49.803371 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 00:59:49.803371 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 00:59:49.803371 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 00:59:49.803371 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 00:59:49.803371 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 00:59:49.803371 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 00:59:49.803371 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 00:59:49.804199 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 00:59:49.804199 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 00:59:49.804199 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 00:59:49.804199 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 00:59:49.804199 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 00:59:49.804199 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 00:59:49.804199 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 00:59:49.804199 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 00:59:49.804199 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 00:59:49.804199 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 00:59:49.804199 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 00:59:49.804199 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 00:59:49.804199 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 135/259] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 8 00:59:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 00:59:49.813472 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aaec3e-4cd5-493c-ba9d-57e20fb34956 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:49.988850 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aaec3e-4cd5-493c-ba9d-57e20fb34956 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:50.031694 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e89a81-50ce-4311-80bd-0355e7d315a9 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:50.061765 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:50.090472 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea33058-1460-4462-996e-1e7faf82afac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:50.091936 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea33058-1460-4462-996e-1e7faf82afac None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:50.092446 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea33058-1460-4462-996e-1e7faf82afac None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:50.093040 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea33058-1460-4462-996e-1e7faf82afac None admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:50.093942 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea33058-1460-4462-996e-1e7faf82afac None admin] RBAC: Adding query filter params (name=admin) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:50.094463 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea33058-1460-4462-996e-1e7faf82afac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:50.096054 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea33058-1460-4462-996e-1e7faf82afac None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:50.100922 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cbd97b-5f95-44f9-a3e5-cd22467a4e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:50.101488 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cbd97b-5f95-44f9-a3e5-cd22467a4e8b None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:50.101734 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cbd97b-5f95-44f9-a3e5-cd22467a4e8b None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:50.102004 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cbd97b-5f95-44f9-a3e5-cd22467a4e8b None admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:50.102679 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cbd97b-5f95-44f9-a3e5-cd22467a4e8b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:50.102983 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cbd97b-5f95-44f9-a3e5-cd22467a4e8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:50.104397 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cbd97b-5f95-44f9-a3e5-cd22467a4e8b None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:50.114254 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 125/260] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 00:59:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:50.117381 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 136/261] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 00:59:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:50.120616 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea33058-1460-4462-996e-1e7faf82afac None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:50.122868 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cbd97b-5f95-44f9-a3e5-cd22467a4e8b None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:50.285864 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:50.334156 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea33058-1460-4462-996e-1e7faf82afac None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:50.359552 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cbd97b-5f95-44f9-a3e5-cd22467a4e8b None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:50.392500 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29444155-746b-4c3d-85b9-bbc9e7f56df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:50.393160 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29444155-746b-4c3d-85b9-bbc9e7f56df8 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:50.393439 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29444155-746b-4c3d-85b9-bbc9e7f56df8 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:50.393712 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29444155-746b-4c3d-85b9-bbc9e7f56df8 None admin] PATH_INFO: `/v3/users/a60b663812334e4a83e64d35179025f6` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:50.395119 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29444155-746b-4c3d-85b9-bbc9e7f56df8 None admin] RBAC: Authorizing `identity:get_user(user_id=a60b663812334e4a83e64d35179025f6)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:50.396605 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29444155-746b-4c3d-85b9-bbc9e7f56df8 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:50.398348 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 126/262] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 8 00:59:50 2022] GET /identity/v3/users/a60b663812334e4a83e64d35179025f6 => generated 283 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:50.405292 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29444155-746b-4c3d-85b9-bbc9e7f56df8 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:50.473010 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d6daef-26e4-454f-93d0-f1f12d7a8fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:50.473678 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d6daef-26e4-454f-93d0-f1f12d7a8fb9 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:50.473956 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d6daef-26e4-454f-93d0-f1f12d7a8fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:50.474228 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d6daef-26e4-454f-93d0-f1f12d7a8fb9 None admin] PATH_INFO: `/v3/users/a60b663812334e4a83e64d35179025f6` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:50.475502 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d6daef-26e4-454f-93d0-f1f12d7a8fb9 None admin] RBAC: Authorizing `identity:get_user(user_id=a60b663812334e4a83e64d35179025f6)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:50.477022 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d6daef-26e4-454f-93d0-f1f12d7a8fb9 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:50.479005 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 137/263] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 8 00:59:50 2022] GET /identity/v3/users/a60b663812334e4a83e64d35179025f6 => generated 283 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 00:59:50.484779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d6daef-26e4-454f-93d0-f1f12d7a8fb9 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:50.523509 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29444155-746b-4c3d-85b9-bbc9e7f56df8 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:50.563552 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f0f3fe-d405-457d-905c-223fdb24b35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:50.564091 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0f3fe-d405-457d-905c-223fdb24b35a None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:50.564391 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0f3fe-d405-457d-905c-223fdb24b35a None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:50.564661 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0f3fe-d405-457d-905c-223fdb24b35a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:50.580502 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:50.595738 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:50.675287 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d6daef-26e4-454f-93d0-f1f12d7a8fb9 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:50.709632 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f0f3fe-d405-457d-905c-223fdb24b35a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:50.711128 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f0f3fe-d405-457d-905c-223fdb24b35a None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:50.735146 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ee194-d5b1-4e14-8eef-b75de625e25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:50.736109 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ee194-d5b1-4e14-8eef-b75de625e25b None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:50.736738 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ee194-d5b1-4e14-8eef-b75de625e25b None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:50.737301 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ee194-d5b1-4e14-8eef-b75de625e25b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:50.874010 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 127/264] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 00:59:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 00:59:50.878793 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f0f3fe-d405-457d-905c-223fdb24b35a None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:50.886681 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85ee194-d5b1-4e14-8eef-b75de625e25b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:50.888731 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85ee194-d5b1-4e14-8eef-b75de625e25b None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:50.927236 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:51.012229 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:51.061488 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f0f3fe-d405-457d-905c-223fdb24b35a None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:51.115176 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 138/265] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 00:59:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 631 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 08 00:59:51.120594 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ee194-d5b1-4e14-8eef-b75de625e25b None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:51.137429 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca4a19d-9ad4-439d-9c6f-25f4f6624d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:51.138116 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca4a19d-9ad4-439d-9c6f-25f4f6624d3d None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:51.138713 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca4a19d-9ad4-439d-9c6f-25f4f6624d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:51.139041 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca4a19d-9ad4-439d-9c6f-25f4f6624d3d None admin] PATH_INFO: `/v3/projects/0f19f058a73f49819b30c33d7fcfc52f` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:51.141727 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca4a19d-9ad4-439d-9c6f-25f4f6624d3d None admin] RBAC: Authorizing `identity:get_project(project_id=0f19f058a73f49819b30c33d7fcfc52f)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:51.143211 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca4a19d-9ad4-439d-9c6f-25f4f6624d3d None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:51.145789 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 128/266] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 00:59:50 2022] GET /identity/v3/projects/0f19f058a73f49819b30c33d7fcfc52f => generated 302 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:51.154166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca4a19d-9ad4-439d-9c6f-25f4f6624d3d None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:51.300533 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ee194-d5b1-4e14-8eef-b75de625e25b None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:51.338933 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddd4863-e30b-4f4a-85cd-a2f155582d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:51.339565 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd4863-e30b-4f4a-85cd-a2f155582d54 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:51.339839 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd4863-e30b-4f4a-85cd-a2f155582d54 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:51.340109 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd4863-e30b-4f4a-85cd-a2f155582d54 None admin] PATH_INFO: `/v3/projects/0f19f058a73f49819b30c33d7fcfc52f` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:51.341819 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddd4863-e30b-4f4a-85cd-a2f155582d54 None admin] RBAC: Authorizing `identity:get_project(project_id=0f19f058a73f49819b30c33d7fcfc52f)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:51.343671 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddd4863-e30b-4f4a-85cd-a2f155582d54 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:51.346326 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 139/267] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 00:59:51 2022] GET /identity/v3/projects/0f19f058a73f49819b30c33d7fcfc52f => generated 302 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 00:59:51.355587 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddd4863-e30b-4f4a-85cd-a2f155582d54 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:51.366243 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca4a19d-9ad4-439d-9c6f-25f4f6624d3d None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:51.397897 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:51.429193 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:51.445134 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b0657-b98b-479f-8cf1-8a62c40e4e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:51.445591 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b0657-b98b-479f-8cf1-8a62c40e4e84 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:51.445923 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b0657-b98b-479f-8cf1-8a62c40e4e84 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:51.446207 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b0657-b98b-479f-8cf1-8a62c40e4e84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:51.446948 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705b0657-b98b-479f-8cf1-8a62c40e4e84 None admin] RBAC: Adding query filter params (role.id=34de369ec7c54715aa83a9052a26c3f2, scope.project.id=0f19f058a73f49819b30c33d7fcfc52f, user.id=a60b663812334e4a83e64d35179025f6) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:51.447259 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705b0657-b98b-479f-8cf1-8a62c40e4e84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:51.448826 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705b0657-b98b-479f-8cf1-8a62c40e4e84 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:51.459174 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 129/268] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 8 00:59:51 2022] GET /identity/v3/role_assignments?user.id=a60b663812334e4a83e64d35179025f6&scope.project.id=0f19f058a73f49819b30c33d7fcfc52f&role.id=34de369ec7c54715aa83a9052a26c3f2 => generated 608 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:51.547715 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddd4863-e30b-4f4a-85cd-a2f155582d54 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:51.591165 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab22075f-728a-4d56-b931-c0a81b5d2c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:51.592524 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab22075f-728a-4d56-b931-c0a81b5d2c1d None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:51.593020 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab22075f-728a-4d56-b931-c0a81b5d2c1d None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:51.593483 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab22075f-728a-4d56-b931-c0a81b5d2c1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:51.594437 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab22075f-728a-4d56-b931-c0a81b5d2c1d None admin] RBAC: Adding query filter params (role.id=069fe3e033664493b3def844e3735bc9, scope.project.id=0f19f058a73f49819b30c33d7fcfc52f, user.id=a60b663812334e4a83e64d35179025f6) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:51.594935 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab22075f-728a-4d56-b931-c0a81b5d2c1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:51.596645 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab22075f-728a-4d56-b931-c0a81b5d2c1d None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:51.603672 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 140/269] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 8 00:59:51 2022] GET /identity/v3/role_assignments?user.id=a60b663812334e4a83e64d35179025f6&scope.project.id=0f19f058a73f49819b30c33d7fcfc52f&role.id=069fe3e033664493b3def844e3735bc9 => generated 608 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:51.984834 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c40ed-f7c1-4e37-88c7-b0a13f81a60c None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:51.985521 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c40ed-f7c1-4e37-88c7-b0a13f81a60c None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:51.986016 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c40ed-f7c1-4e37-88c7-b0a13f81a60c None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:51.987804 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 130/270] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 00:59:51.993631 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b2f984-948f-45a2-9093-6398d5a0b8a9 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:51.994184 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b2f984-948f-45a2-9093-6398d5a0b8a9 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:51.994649 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b2f984-948f-45a2-9093-6398d5a0b8a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:52.017378 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-97b2f984-948f-45a2-9093-6398d5a0b8a9 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:52.018897 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-97b2f984-948f-45a2-9093-6398d5a0b8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:52.087566 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e4c16f-42e8-4921-87e9-0c0a2601de55 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:52.088062 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e4c16f-42e8-4921-87e9-0c0a2601de55 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:52.088164 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e4c16f-42e8-4921-87e9-0c0a2601de55 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:52.089600 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 131/271] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 00:59:52.096451 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c301964-1dcc-4f93-89c1-4dae1952ce2b None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:52.096737 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c301964-1dcc-4f93-89c1-4dae1952ce2b None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:52.096993 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c301964-1dcc-4f93-89c1-4dae1952ce2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:52.098149 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 141/272] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 00:59:52.129146 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-0c301964-1dcc-4f93-89c1-4dae1952ce2b None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:52.130346 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-0c301964-1dcc-4f93-89c1-4dae1952ce2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:52.226092 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 132/273] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 00:59:52.333112 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4722b0-7497-41b6-9077-e0279c54f068 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:52.334076 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4722b0-7497-41b6-9077-e0279c54f068 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:52.334554 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4722b0-7497-41b6-9077-e0279c54f068 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:52.336463 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 142/274] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 00:59:52.341621 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487debc0-9127-43bf-9ed4-9f89f7b9a5fe None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:52.341907 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487debc0-9127-43bf-9ed4-9f89f7b9a5fe None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:52.342379 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487debc0-9127-43bf-9ed4-9f89f7b9a5fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:52.367366 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-487debc0-9127-43bf-9ed4-9f89f7b9a5fe None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:52.368945 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-487debc0-9127-43bf-9ed4-9f89f7b9a5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:52.399454 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4722b0-7497-41b6-9077-e0279c54f068 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:52.423954 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:52.453060 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 133/275] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 00:59:52.520060 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4722b0-7497-41b6-9077-e0279c54f068 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:52.530545 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487debc0-9127-43bf-9ed4-9f89f7b9a5fe None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:52.563664 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5ffba2-d9c2-490b-af76-e97d0f4dfe0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:52.564364 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ffba2-d9c2-490b-af76-e97d0f4dfe0e None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:52.564648 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ffba2-d9c2-490b-af76-e97d0f4dfe0e None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:52.564927 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ffba2-d9c2-490b-af76-e97d0f4dfe0e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:52.574990 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:52.581245 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5ffba2-d9c2-490b-af76-e97d0f4dfe0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 00:59:52.587779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-da5ffba2-d9c2-490b-af76-e97d0f4dfe0e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 00:59:52.587779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 00:59:52.587779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 00:59:52.587779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 00:59:52.587779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 00:59:52.587779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 00:59:52.587779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 00:59:52.587779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 00:59:52.587779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 00:59:52.587779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 00:59:52.587779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 00:59:52.587779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 00:59:52.587779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 00:59:52.587779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 00:59:52.587779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 00:59:52.587779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 00:59:52.588697 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 00:59:52.588697 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 00:59:52.588697 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 00:59:52.588697 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 00:59:52.588697 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 00:59:52.588697 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 00:59:52.588697 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 00:59:52.588697 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 00:59:52.588697 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 00:59:52.588697 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 00:59:52.588697 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 00:59:52.588697 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 00:59:52.588697 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 00:59:52.588697 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 00:59:52.588697 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 00:59:52.588697 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 00:59:52.589494 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 00:59:52.589494 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 00:59:52.589494 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 00:59:52.589494 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 00:59:52.589494 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 00:59:52.589494 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 00:59:52.589494 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 00:59:52.589494 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 00:59:52.589494 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 00:59:52.589494 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 00:59:52.589494 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 00:59:52.589494 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 00:59:52.589494 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 143/276] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 00:59:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 00:59:52.596860 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5ffba2-d9c2-490b-af76-e97d0f4dfe0e None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:52.684042 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487debc0-9127-43bf-9ed4-9f89f7b9a5fe None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:52.691411 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:52.715384 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5ffba2-d9c2-490b-af76-e97d0f4dfe0e None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:52.721721 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ee61e-971e-40cd-bcde-23000fd7c1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:52.722609 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ee61e-971e-40cd-bcde-23000fd7c1da None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:52.723577 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ee61e-971e-40cd-bcde-23000fd7c1da None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:52.724033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ee61e-971e-40cd-bcde-23000fd7c1da None admin] PATH_INFO: `/v3/roles/member` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:52.737745 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3ee61e-971e-40cd-bcde-23000fd7c1da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 00:59:52.742605 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-4d3ee61e-971e-40cd-bcde-23000fd7c1da None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 00:59:52.742605 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 00:59:52.742605 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 00:59:52.742605 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 00:59:52.742605 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 00:59:52.742605 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 00:59:52.742605 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 00:59:52.742605 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 00:59:52.742605 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 00:59:52.742605 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 00:59:52.742605 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 00:59:52.742605 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 00:59:52.742605 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 00:59:52.742605 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 00:59:52.742605 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 00:59:52.742605 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 00:59:52.743568 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 00:59:52.743568 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 00:59:52.743568 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 00:59:52.743568 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 00:59:52.743568 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 00:59:52.743568 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 00:59:52.743568 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 00:59:52.743568 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 00:59:52.743568 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 00:59:52.743568 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 00:59:52.743568 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 00:59:52.743568 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 00:59:52.743568 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 00:59:52.743568 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 00:59:52.743568 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 00:59:52.743568 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 00:59:52.744384 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 00:59:52.744384 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 00:59:52.744384 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 00:59:52.744384 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 00:59:52.744384 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 00:59:52.744384 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 00:59:52.744384 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 00:59:52.744384 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 00:59:52.744384 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 00:59:52.744384 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 00:59:52.744384 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 00:59:52.744384 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 00:59:52.744384 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 134/277] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 00:59:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 00:59:52.745373 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db074083-2925-4dbb-acb7-2fcd320c5ce2 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:52.745609 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db074083-2925-4dbb-acb7-2fcd320c5ce2 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:52.745871 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db074083-2925-4dbb-acb7-2fcd320c5ce2 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:52.747106 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 135/278] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 00:59:52.752212 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db074083-2925-4dbb-acb7-2fcd320c5ce2 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:52.757825 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3110f742-5d4b-4cde-8577-506675db75c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:52.758434 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3110f742-5d4b-4cde-8577-506675db75c2 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:52.758713 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3110f742-5d4b-4cde-8577-506675db75c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:52.758983 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3110f742-5d4b-4cde-8577-506675db75c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:52.759650 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3110f742-5d4b-4cde-8577-506675db75c2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:52.759956 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3110f742-5d4b-4cde-8577-506675db75c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:52.761648 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3110f742-5d4b-4cde-8577-506675db75c2 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:52.770527 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 144/279] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 8 00:59:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 00:59:52.773049 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3110f742-5d4b-4cde-8577-506675db75c2 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:52.843084 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:52.850517 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db074083-2925-4dbb-acb7-2fcd320c5ce2 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:52.871374 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3110f742-5d4b-4cde-8577-506675db75c2 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:52.883719 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd2c243-9c62-4032-8449-d691a52ecbea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:52.884376 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd2c243-9c62-4032-8449-d691a52ecbea None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:52.884656 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd2c243-9c62-4032-8449-d691a52ecbea None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:52.884926 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd2c243-9c62-4032-8449-d691a52ecbea None admin] PATH_INFO: `/v3/domains/default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:52.886271 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd2c243-9c62-4032-8449-d691a52ecbea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:52.887751 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd2c243-9c62-4032-8449-d691a52ecbea None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:52.889898 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 136/280] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 00:59:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:52.890987 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ae864-7247-4233-8700-722f872315bb None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:52.891267 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ae864-7247-4233-8700-722f872315bb None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:52.891536 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ae864-7247-4233-8700-722f872315bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:52.905835 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d68c21d-71b9-403d-8798-b34697713e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:52.906497 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d68c21d-71b9-403d-8798-b34697713e6b None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:52.906771 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d68c21d-71b9-403d-8798-b34697713e6b None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:52.907037 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d68c21d-71b9-403d-8798-b34697713e6b None admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:52.907687 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d68c21d-71b9-403d-8798-b34697713e6b None admin] RBAC: Adding query filter params (name=member) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:52.907983 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d68c21d-71b9-403d-8798-b34697713e6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:52.909390 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d68c21d-71b9-403d-8798-b34697713e6b None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:52.909488 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-7e1ae864-7247-4233-8700-722f872315bb None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:52.910617 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-7e1ae864-7247-4233-8700-722f872315bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:52.917496 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 145/281] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 8 00:59:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:52.918070 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d68c21d-71b9-403d-8798-b34697713e6b None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:52.920806 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:52.982032 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 137/282] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 00:59:52.982852 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1ae864-7247-4233-8700-722f872315bb None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:53.023550 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d68c21d-71b9-403d-8798-b34697713e6b None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:53.058420 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ff5f12-97fe-4c98-a68b-7c6824d532e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:53.059046 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff5f12-97fe-4c98-a68b-7c6824d532e7 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:53.059322 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff5f12-97fe-4c98-a68b-7c6824d532e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:53.059595 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff5f12-97fe-4c98-a68b-7c6824d532e7 None admin] PATH_INFO: `/v3/users/ca8b7995da7e4dfaa839b2c9579a9577` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:53.061311 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ff5f12-97fe-4c98-a68b-7c6824d532e7 None admin] RBAC: Authorizing `identity:get_user(user_id=ca8b7995da7e4dfaa839b2c9579a9577)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:53.062773 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ff5f12-97fe-4c98-a68b-7c6824d532e7 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:53.064352 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 146/283] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 8 00:59:52 2022] GET /identity/v3/users/ca8b7995da7e4dfaa839b2c9579a9577 => generated 297 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:53.065093 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ff5f12-97fe-4c98-a68b-7c6824d532e7 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:53.098708 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:53.107131 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1ae864-7247-4233-8700-722f872315bb None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:53.141776 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e29053-d1ed-4059-800a-41e78aaa0035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:53.142416 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e29053-d1ed-4059-800a-41e78aaa0035 None admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:53.142695 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e29053-d1ed-4059-800a-41e78aaa0035 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:53.142967 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e29053-d1ed-4059-800a-41e78aaa0035 None admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:53.143609 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e29053-d1ed-4059-800a-41e78aaa0035 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:53.145090 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e29053-d1ed-4059-800a-41e78aaa0035 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:53.148975 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:53.167234 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 138/284] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 8 00:59:52 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 00:59:53.177542 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e29053-d1ed-4059-800a-41e78aaa0035 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:53.192446 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ff5f12-97fe-4c98-a68b-7c6824d532e7 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:53.237018 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81865cf-a411-434f-880d-e6ec20ab5ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:53.237821 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81865cf-a411-434f-880d-e6ec20ab5ac7 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:53.238210 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81865cf-a411-434f-880d-e6ec20ab5ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:53.238614 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81865cf-a411-434f-880d-e6ec20ab5ac7 None admin] PATH_INFO: `/v3/users/36d836f65f724326a9a44c4c6e87d79a` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:53.255909 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81865cf-a411-434f-880d-e6ec20ab5ac7 None admin] RBAC: Authorizing `identity:get_user(user_id=36d836f65f724326a9a44c4c6e87d79a)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:53.257468 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81865cf-a411-434f-880d-e6ec20ab5ac7 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:53.259471 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 147/285] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 8 00:59:53 2022] GET /identity/v3/users/36d836f65f724326a9a44c4c6e87d79a => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:53.260169 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81865cf-a411-434f-880d-e6ec20ab5ac7 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:53.319775 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e29053-d1ed-4059-800a-41e78aaa0035 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:53.361673 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507a81c8-9bf8-4b54-8c82-09a349defa42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:53.362238 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507a81c8-9bf8-4b54-8c82-09a349defa42 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:53.362735 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507a81c8-9bf8-4b54-8c82-09a349defa42 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:53.363008 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507a81c8-9bf8-4b54-8c82-09a349defa42 None admin] PATH_INFO: `/v3/projects/0f19f058a73f49819b30c33d7fcfc52f` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:53.364244 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507a81c8-9bf8-4b54-8c82-09a349defa42 None admin] RBAC: Authorizing `identity:get_project(project_id=0f19f058a73f49819b30c33d7fcfc52f)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:53.365723 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507a81c8-9bf8-4b54-8c82-09a349defa42 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:53.367790 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 139/286] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 00:59:53 2022] GET /identity/v3/projects/0f19f058a73f49819b30c33d7fcfc52f => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:53.368238 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507a81c8-9bf8-4b54-8c82-09a349defa42 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:53.386783 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81865cf-a411-434f-880d-e6ec20ab5ac7 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:53.398646 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:53.417211 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:53.426752 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036d41f2-4ee3-453c-8478-fd0b722ea4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:53.427395 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d41f2-4ee3-453c-8478-fd0b722ea4a5 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:53.427686 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d41f2-4ee3-453c-8478-fd0b722ea4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:53.427955 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d41f2-4ee3-453c-8478-fd0b722ea4a5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:53.445651 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036d41f2-4ee3-453c-8478-fd0b722ea4a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 00:59:53.451615 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-036d41f2-4ee3-453c-8478-fd0b722ea4a5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 00:59:53.451615 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 00:59:53.451615 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 00:59:53.451615 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 00:59:53.451615 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 00:59:53.451615 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 00:59:53.451615 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 00:59:53.451615 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 00:59:53.451615 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 00:59:53.451615 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 00:59:53.451615 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 00:59:53.451615 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 00:59:53.451615 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 00:59:53.451615 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 00:59:53.451615 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 00:59:53.451615 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 00:59:53.452620 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 00:59:53.452620 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 00:59:53.452620 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 00:59:53.452620 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 00:59:53.452620 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 00:59:53.452620 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 00:59:53.452620 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 00:59:53.452620 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 00:59:53.452620 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 00:59:53.452620 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 00:59:53.452620 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 00:59:53.452620 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 00:59:53.452620 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 00:59:53.452620 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 00:59:53.452620 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 00:59:53.452620 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 00:59:53.453471 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 00:59:53.453471 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 00:59:53.453471 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 00:59:53.453471 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 00:59:53.453471 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 00:59:53.453471 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 00:59:53.453471 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 00:59:53.453471 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 00:59:53.453471 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 00:59:53.453471 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 00:59:53.453471 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 00:59:53.453471 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 00:59:53.455460 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 148/287] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 00:59:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 00:59:53.456949 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036d41f2-4ee3-453c-8478-fd0b722ea4a5 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:53.505047 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507a81c8-9bf8-4b54-8c82-09a349defa42 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:53.546555 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d6f156-6282-4591-9c10-0aea68ba7ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:53.547174 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d6f156-6282-4591-9c10-0aea68ba7ee4 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:53.547456 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d6f156-6282-4591-9c10-0aea68ba7ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:53.547733 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d6f156-6282-4591-9c10-0aea68ba7ee4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:53.548520 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d6f156-6282-4591-9c10-0aea68ba7ee4 None admin] RBAC: Adding query filter params (role.id=92aa6d6824404d10b4a3be8b4a6bc317, user.id=36d836f65f724326a9a44c4c6e87d79a, scope.system=all) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:53.548834 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d6f156-6282-4591-9c10-0aea68ba7ee4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:53.550300 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d6f156-6282-4591-9c10-0aea68ba7ee4 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:53.555877 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 140/288] 10.222.0.42 () {56 vars in 1308 bytes} [Thu Sep 8 00:59:53 2022] GET /identity/v3/role_assignments?user.id=36d836f65f724326a9a44c4c6e87d79a&scope.system=all&role.id=92aa6d6824404d10b4a3be8b4a6bc317 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:53.557616 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d6f156-6282-4591-9c10-0aea68ba7ee4 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:53.590761 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036d41f2-4ee3-453c-8478-fd0b722ea4a5 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:53.651723 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ab610d-9311-45d6-bafc-1809083abb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:53.651723 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab610d-9311-45d6-bafc-1809083abb39 None admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:53.651723 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab610d-9311-45d6-bafc-1809083abb39 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:53.651723 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab610d-9311-45d6-bafc-1809083abb39 None admin] PATH_INFO: `/v3/projects/0f19f058a73f49819b30c33d7fcfc52f/users/ca8b7995da7e4dfaa839b2c9579a9577/roles/653150c294214a0da3bdd8d00aacd3aa` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:53.654355 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ab610d-9311-45d6-bafc-1809083abb39 None admin] RBAC: Authorizing `identity:create_grant(project_id=0f19f058a73f49819b30c33d7fcfc52f, user_id=ca8b7995da7e4dfaa839b2c9579a9577, role_id=653150c294214a0da3bdd8d00aacd3aa)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:53.655818 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ab610d-9311-45d6-bafc-1809083abb39 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:53.673256 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 149/289] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 8 00:59:53 2022] PUT /identity/v3/projects/0f19f058a73f49819b30c33d7fcfc52f/users/ca8b7995da7e4dfaa839b2c9579a9577/roles/653150c294214a0da3bdd8d00aacd3aa => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 00:59:53.713679 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:53.749496 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d6f156-6282-4591-9c10-0aea68ba7ee4 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:53.809578 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefe8f5d-bdc7-48ba-bb74-5e8b40b752a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:53.810448 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe8f5d-bdc7-48ba-bb74-5e8b40b752a9 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:53.810727 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe8f5d-bdc7-48ba-bb74-5e8b40b752a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:53.810999 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe8f5d-bdc7-48ba-bb74-5e8b40b752a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:53.811694 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefe8f5d-bdc7-48ba-bb74-5e8b40b752a9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:53.811998 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefe8f5d-bdc7-48ba-bb74-5e8b40b752a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:53.813463 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefe8f5d-bdc7-48ba-bb74-5e8b40b752a9 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:53.823984 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 141/290] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 8 00:59:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:53.830609 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ab610d-9311-45d6-bafc-1809083abb39 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:53.907151 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:53.971394 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ab610d-9311-45d6-bafc-1809083abb39 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:54.008108 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c1b651-2026-4253-8494-be8763397b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:54.008771 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c1b651-2026-4253-8494-be8763397b9a None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:54.009042 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c1b651-2026-4253-8494-be8763397b9a None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:54.009308 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c1b651-2026-4253-8494-be8763397b9a None admin] PATH_INFO: `/v3/users/3169b2a903354aa08f4b4415b53d39e2` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:54.010441 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c1b651-2026-4253-8494-be8763397b9a None admin] RBAC: Authorizing `identity:get_user(user_id=3169b2a903354aa08f4b4415b53d39e2)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:54.012483 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c1b651-2026-4253-8494-be8763397b9a None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:54.015521 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 150/291] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 8 00:59:53 2022] GET /identity/v3/users/3169b2a903354aa08f4b4415b53d39e2 => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:54.020888 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefe8f5d-bdc7-48ba-bb74-5e8b40b752a9 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:54.136025 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefe8f5d-bdc7-48ba-bb74-5e8b40b752a9 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:54.183314 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eccbbf-6bf7-480d-9ab3-5244dcdd87d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:54.184288 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eccbbf-6bf7-480d-9ab3-5244dcdd87d2 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:54.185302 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eccbbf-6bf7-480d-9ab3-5244dcdd87d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:54.185775 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eccbbf-6bf7-480d-9ab3-5244dcdd87d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:54.201982 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:54.280537 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eccbbf-6bf7-480d-9ab3-5244dcdd87d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:54.282076 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eccbbf-6bf7-480d-9ab3-5244dcdd87d2 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:54.418622 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 142/292] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 00:59:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 00:59:54.424350 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c1b651-2026-4253-8494-be8763397b9a None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:54.556630 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c1b651-2026-4253-8494-be8763397b9a None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:54.584515 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:54.594044 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3480a7d-704b-4a45-a475-81a706890207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:54.594677 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3480a7d-704b-4a45-a475-81a706890207 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:54.594954 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3480a7d-704b-4a45-a475-81a706890207 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:54.595221 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3480a7d-704b-4a45-a475-81a706890207 None admin] PATH_INFO: `/v3/projects/0f19f058a73f49819b30c33d7fcfc52f` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:54.596381 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3480a7d-704b-4a45-a475-81a706890207 None admin] RBAC: Authorizing `identity:get_project(project_id=0f19f058a73f49819b30c33d7fcfc52f)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:54.597824 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3480a7d-704b-4a45-a475-81a706890207 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:54.599735 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 151/293] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 00:59:54 2022] GET /identity/v3/projects/0f19f058a73f49819b30c33d7fcfc52f => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:54.609210 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eccbbf-6bf7-480d-9ab3-5244dcdd87d2 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:54.682730 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:54.765046 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eccbbf-6bf7-480d-9ab3-5244dcdd87d2 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:54.819716 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f9a663-12a7-4327-ba82-c0b3444d1102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:54.820276 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9a663-12a7-4327-ba82-c0b3444d1102 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:54.820555 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9a663-12a7-4327-ba82-c0b3444d1102 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:54.820825 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9a663-12a7-4327-ba82-c0b3444d1102 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:54.821605 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f9a663-12a7-4327-ba82-c0b3444d1102 None admin] RBAC: Adding query filter params (role.id=92aa6d6824404d10b4a3be8b4a6bc317, scope.project.id=0f19f058a73f49819b30c33d7fcfc52f, user.id=3169b2a903354aa08f4b4415b53d39e2) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:54.821863 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f9a663-12a7-4327-ba82-c0b3444d1102 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:54.823323 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f9a663-12a7-4327-ba82-c0b3444d1102 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:54.832869 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 143/294] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 8 00:59:54 2022] GET /identity/v3/role_assignments?user.id=3169b2a903354aa08f4b4415b53d39e2&scope.project.id=0f19f058a73f49819b30c33d7fcfc52f&role.id=92aa6d6824404d10b4a3be8b4a6bc317 => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:55.789920 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d3d88-854d-4e17-a91e-b205f0407742 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:55.790431 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d3d88-854d-4e17-a91e-b205f0407742 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:55.790503 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d3d88-854d-4e17-a91e-b205f0407742 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:55.792348 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 152/295] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 00:59:55.798511 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fbbb97-abe3-4f3b-ac82-6eccd01c6692 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:55.799044 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fbbb97-abe3-4f3b-ac82-6eccd01c6692 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:55.799044 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fbbb97-abe3-4f3b-ac82-6eccd01c6692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:55.818966 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-58fbbb97-abe3-4f3b-ac82-6eccd01c6692 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:55.820241 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-58fbbb97-abe3-4f3b-ac82-6eccd01c6692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:55.890296 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 144/296] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 00:59:55.937651 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cee73e-1f33-4a65-9491-c50ef2148693 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:55.938274 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cee73e-1f33-4a65-9491-c50ef2148693 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:55.938767 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cee73e-1f33-4a65-9491-c50ef2148693 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:55.940335 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 153/297] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 00:59:55.945793 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6439924-5002-4619-a5be-6de4456953c2 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:55.946299 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6439924-5002-4619-a5be-6de4456953c2 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:55.946795 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6439924-5002-4619-a5be-6de4456953c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:55.966828 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-f6439924-5002-4619-a5be-6de4456953c2 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:55.968495 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-f6439924-5002-4619-a5be-6de4456953c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:56.003604 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:56.043326 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 145/298] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 00:59:56.087311 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68fff0-2216-478c-8b0e-ad7cd22f65d5 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:56.087916 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68fff0-2216-478c-8b0e-ad7cd22f65d5 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:56.088434 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68fff0-2216-478c-8b0e-ad7cd22f65d5 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:56.090291 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 154/299] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 00:59:56.097450 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39644514-754a-479d-8590-cea36b19e7c7 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:56.097972 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39644514-754a-479d-8590-cea36b19e7c7 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:56.098463 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39644514-754a-479d-8590-cea36b19e7c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:56.117468 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-39644514-754a-479d-8590-cea36b19e7c7 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:56.118915 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-39644514-754a-479d-8590-cea36b19e7c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:56.167472 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e68fff0-2216-478c-8b0e-ad7cd22f65d5 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:56.195214 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 146/300] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 00:59:56.279581 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e68fff0-2216-478c-8b0e-ad7cd22f65d5 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:56.315696 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da23b218-7e96-4744-83b4-a057672c4126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:56.316347 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da23b218-7e96-4744-83b4-a057672c4126 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:56.316666 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da23b218-7e96-4744-83b4-a057672c4126 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:56.316940 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da23b218-7e96-4744-83b4-a057672c4126 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:56.323593 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39644514-754a-479d-8590-cea36b19e7c7 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:56.329194 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da23b218-7e96-4744-83b4-a057672c4126 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 00:59:56.334544 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-da23b218-7e96-4744-83b4-a057672c4126 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 00:59:56.334544 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 00:59:56.334544 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 00:59:56.334544 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 00:59:56.334544 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 00:59:56.334544 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 00:59:56.334544 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 00:59:56.334544 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 00:59:56.334544 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 00:59:56.334544 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 00:59:56.334544 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 00:59:56.334544 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 00:59:56.334544 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 00:59:56.334544 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 00:59:56.334544 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 00:59:56.334544 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 00:59:56.335498 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 00:59:56.335498 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 00:59:56.335498 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 00:59:56.335498 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 00:59:56.335498 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 00:59:56.335498 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 00:59:56.335498 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 00:59:56.335498 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 00:59:56.335498 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 00:59:56.335498 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 00:59:56.335498 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 00:59:56.335498 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 00:59:56.335498 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 00:59:56.335498 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 00:59:56.335498 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 00:59:56.335498 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 00:59:56.336336 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 00:59:56.336336 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 00:59:56.336336 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 00:59:56.336336 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 00:59:56.336336 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 00:59:56.336336 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 00:59:56.336336 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 00:59:56.336336 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 00:59:56.336336 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 00:59:56.336336 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 00:59:56.336336 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 00:59:56.336336 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 00:59:56.336942 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 155/301] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 00:59:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 00:59:56.345114 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da23b218-7e96-4744-83b4-a057672c4126 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:56.413569 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:56.446933 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:56.453480 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39644514-754a-479d-8590-cea36b19e7c7 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:56.470807 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da23b218-7e96-4744-83b4-a057672c4126 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:56.492099 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fd9fba-a03e-4fb5-885d-b431455f8b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:56.492760 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd9fba-a03e-4fb5-885d-b431455f8b39 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:56.493040 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd9fba-a03e-4fb5-885d-b431455f8b39 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:56.493310 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd9fba-a03e-4fb5-885d-b431455f8b39 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:56.494413 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fd9fba-a03e-4fb5-885d-b431455f8b39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:56.495891 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fd9fba-a03e-4fb5-885d-b431455f8b39 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:56.497618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 147/302] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 00:59:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 00:59:56.498788 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c720d9-2a30-49fd-b085-e084629bb5f7 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:56.499073 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c720d9-2a30-49fd-b085-e084629bb5f7 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:56.499345 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c720d9-2a30-49fd-b085-e084629bb5f7 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:56.504316 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 148/303] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 00:59:56.504316 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c720d9-2a30-49fd-b085-e084629bb5f7 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:56.509853 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55168e4-1369-423d-a34a-d302e84808e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:56.510454 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55168e4-1369-423d-a34a-d302e84808e6 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:56.510984 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55168e4-1369-423d-a34a-d302e84808e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:56.511248 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55168e4-1369-423d-a34a-d302e84808e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:56.511907 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55168e4-1369-423d-a34a-d302e84808e6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:56.512235 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55168e4-1369-423d-a34a-d302e84808e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:56.513646 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55168e4-1369-423d-a34a-d302e84808e6 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:56.521812 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 156/304] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 8 00:59:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 00:59:56.522558 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55168e4-1369-423d-a34a-d302e84808e6 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:56.605991 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c720d9-2a30-49fd-b085-e084629bb5f7 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:56.621456 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55168e4-1369-423d-a34a-d302e84808e6 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:56.628800 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:56.643648 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e60d6c6-a04e-4de3-92cd-cf7b3c3c487a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:56.644525 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e60d6c6-a04e-4de3-92cd-cf7b3c3c487a None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:56.644804 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e60d6c6-a04e-4de3-92cd-cf7b3c3c487a None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:56.645077 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e60d6c6-a04e-4de3-92cd-cf7b3c3c487a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:56.655981 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e60d6c6-a04e-4de3-92cd-cf7b3c3c487a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 00:59:56.657851 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccf86c2-e616-4012-925f-a7969dc22a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:56.658460 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf86c2-e616-4012-925f-a7969dc22a1d None admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:56.658743 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf86c2-e616-4012-925f-a7969dc22a1d None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:56.659021 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf86c2-e616-4012-925f-a7969dc22a1d None admin] PATH_INFO: `/v3/groups` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:56.659654 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccf86c2-e616-4012-925f-a7969dc22a1d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:56.660854 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-2e60d6c6-a04e-4de3-92cd-cf7b3c3c487a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 00:59:56.660854 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 00:59:56.660854 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 00:59:56.660854 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 00:59:56.660854 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 00:59:56.660854 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 00:59:56.660854 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 00:59:56.660854 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 00:59:56.660854 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 00:59:56.660854 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 00:59:56.660854 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 00:59:56.660854 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 00:59:56.660854 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 00:59:56.660854 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 00:59:56.660854 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 00:59:56.660854 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 00:59:56.661957 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 00:59:56.661957 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 00:59:56.661957 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 00:59:56.661957 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 00:59:56.661957 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 00:59:56.661957 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 00:59:56.661957 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 00:59:56.661957 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 00:59:56.661957 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 00:59:56.661957 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 00:59:56.661957 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 00:59:56.661957 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 00:59:56.661957 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 00:59:56.661957 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 00:59:56.661957 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 00:59:56.661957 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 00:59:56.662768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 00:59:56.662768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 00:59:56.662768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 00:59:56.662768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 00:59:56.662768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 00:59:56.662768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 00:59:56.662768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 00:59:56.662768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 00:59:56.662768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 00:59:56.662768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 00:59:56.662768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 00:59:56.662768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 00:59:56.663470 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccf86c2-e616-4012-925f-a7969dc22a1d None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:56.663537 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 149/305] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 00:59:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 00:59:56.663537 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16993006-2b24-47a4-a5e8-578ff8a8aa15 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:56.663815 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16993006-2b24-47a4-a5e8-578ff8a8aa15 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:56.664089 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16993006-2b24-47a4-a5e8-578ff8a8aa15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:56.677779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 157/306] 10.222.0.42 () {60 vars in 1134 bytes} [Thu Sep 8 00:59:56 2022] POST /identity/v3/groups => generated 229 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 00:59:56.678547 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccf86c2-e616-4012-925f-a7969dc22a1d None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:56.686326 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-16993006-2b24-47a4-a5e8-578ff8a8aa15 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:56.687574 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-16993006-2b24-47a4-a5e8-578ff8a8aa15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:56.714103 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:56.769019 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 150/307] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 00:59:56.769899 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16993006-2b24-47a4-a5e8-578ff8a8aa15 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:56.799242 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccf86c2-e616-4012-925f-a7969dc22a1d None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:56.847137 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d62dd1-6a46-4507-bbd2-98c2e8741d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:56.848225 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d62dd1-6a46-4507-bbd2-98c2e8741d8f None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:56.848508 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d62dd1-6a46-4507-bbd2-98c2e8741d8f None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:56.848777 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d62dd1-6a46-4507-bbd2-98c2e8741d8f None admin] PATH_INFO: `/v3/users/640b55004f3f4278b286d4f1e6ef2132` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:56.867948 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d62dd1-6a46-4507-bbd2-98c2e8741d8f None admin] RBAC: Authorizing `identity:get_user(user_id=640b55004f3f4278b286d4f1e6ef2132)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:56.870138 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d62dd1-6a46-4507-bbd2-98c2e8741d8f None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:56.871810 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 158/308] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 8 00:59:56 2022] GET /identity/v3/users/640b55004f3f4278b286d4f1e6ef2132 => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:56.877651 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d62dd1-6a46-4507-bbd2-98c2e8741d8f None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:56.913824 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:56.926287 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16993006-2b24-47a4-a5e8-578ff8a8aa15 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:56.967762 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74edec61-4d40-4400-a2c1-f13a06711d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:56.968904 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74edec61-4d40-4400-a2c1-f13a06711d32 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:56.970280 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74edec61-4d40-4400-a2c1-f13a06711d32 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:56.970709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74edec61-4d40-4400-a2c1-f13a06711d32 None admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:56.971581 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74edec61-4d40-4400-a2c1-f13a06711d32 None admin] RBAC: Adding query filter params (name=member) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:56.972030 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74edec61-4d40-4400-a2c1-f13a06711d32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:56.973589 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74edec61-4d40-4400-a2c1-f13a06711d32 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:56.986545 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 151/309] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 8 00:59:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:56.992730 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74edec61-4d40-4400-a2c1-f13a06711d32 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:57.006844 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d62dd1-6a46-4507-bbd2-98c2e8741d8f None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:57.013467 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:57.051977 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a59e3f-2fb0-4c2e-80bc-41d4e5b35de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:57.052621 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a59e3f-2fb0-4c2e-80bc-41d4e5b35de6 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:57.052904 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a59e3f-2fb0-4c2e-80bc-41d4e5b35de6 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:57.053179 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a59e3f-2fb0-4c2e-80bc-41d4e5b35de6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:57.053909 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a59e3f-2fb0-4c2e-80bc-41d4e5b35de6 None admin] RBAC: Adding query filter params (role.id=653150c294214a0da3bdd8d00aacd3aa, user.id=640b55004f3f4278b286d4f1e6ef2132, scope.system=all) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:57.054224 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a59e3f-2fb0-4c2e-80bc-41d4e5b35de6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:57.055664 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a59e3f-2fb0-4c2e-80bc-41d4e5b35de6 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:57.061285 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 159/310] 10.222.0.42 () {56 vars in 1308 bytes} [Thu Sep 8 00:59:56 2022] GET /identity/v3/role_assignments?user.id=640b55004f3f4278b286d4f1e6ef2132&scope.system=all&role.id=653150c294214a0da3bdd8d00aacd3aa => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 00:59:57.061911 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a59e3f-2fb0-4c2e-80bc-41d4e5b35de6 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:57.122331 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74edec61-4d40-4400-a2c1-f13a06711d32 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:57.170427 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e3cb75-2710-4afc-aa70-6b8a7df4aa72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:57.171097 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3cb75-2710-4afc-aa70-6b8a7df4aa72 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:57.171409 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3cb75-2710-4afc-aa70-6b8a7df4aa72 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:57.172021 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3cb75-2710-4afc-aa70-6b8a7df4aa72 None admin] PATH_INFO: `/v3/users/36d836f65f724326a9a44c4c6e87d79a` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:57.173570 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e3cb75-2710-4afc-aa70-6b8a7df4aa72 None admin] RBAC: Authorizing `identity:get_user(user_id=36d836f65f724326a9a44c4c6e87d79a)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:57.175253 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e3cb75-2710-4afc-aa70-6b8a7df4aa72 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:57.177861 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 152/311] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 8 00:59:56 2022] GET /identity/v3/users/36d836f65f724326a9a44c4c6e87d79a => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 00:59:57.182845 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:57.184085 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e3cb75-2710-4afc-aa70-6b8a7df4aa72 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:57.204551 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a59e3f-2fb0-4c2e-80bc-41d4e5b35de6 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:57.249828 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd162d4-151d-4f2c-88f3-62f7f4f1d541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:57.250476 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd162d4-151d-4f2c-88f3-62f7f4f1d541 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:57.250777 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd162d4-151d-4f2c-88f3-62f7f4f1d541 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:57.251046 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd162d4-151d-4f2c-88f3-62f7f4f1d541 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:57.264312 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd162d4-151d-4f2c-88f3-62f7f4f1d541 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 00:59:57.270396 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-9fd162d4-151d-4f2c-88f3-62f7f4f1d541 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 00:59:57.270396 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 00:59:57.270396 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 00:59:57.270396 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 00:59:57.270396 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 00:59:57.270396 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 00:59:57.270396 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 00:59:57.270396 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 00:59:57.270396 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 00:59:57.270396 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 00:59:57.270396 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 00:59:57.270396 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 00:59:57.270396 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 00:59:57.270396 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 00:59:57.270396 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 00:59:57.270396 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 00:59:57.271484 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 00:59:57.271484 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 00:59:57.271484 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 00:59:57.271484 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 00:59:57.271484 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 00:59:57.271484 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 00:59:57.271484 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 00:59:57.271484 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 00:59:57.271484 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 00:59:57.271484 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 00:59:57.271484 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 00:59:57.271484 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 00:59:57.271484 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 00:59:57.271484 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 00:59:57.271484 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 00:59:57.271484 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 00:59:57.272453 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 00:59:57.272453 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 00:59:57.272453 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 00:59:57.272453 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 00:59:57.272453 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 00:59:57.272453 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 00:59:57.272453 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 00:59:57.272453 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 00:59:57.272453 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 00:59:57.272453 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 00:59:57.272453 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 00:59:57.272453 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 00:59:57.272453 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 160/312] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 00:59:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 00:59:57.277523 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd162d4-151d-4f2c-88f3-62f7f4f1d541 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:57.286694 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:57.303980 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e3cb75-2710-4afc-aa70-6b8a7df4aa72 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:57.344510 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f593bc7-bdaf-4981-a589-94630a7005e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:57.345264 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f593bc7-bdaf-4981-a589-94630a7005e9 None admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:57.345720 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f593bc7-bdaf-4981-a589-94630a7005e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:57.346071 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f593bc7-bdaf-4981-a589-94630a7005e9 None admin] PATH_INFO: `/v3/system/users/36d836f65f724326a9a44c4c6e87d79a/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:57.348628 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f593bc7-bdaf-4981-a589-94630a7005e9 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=36d836f65f724326a9a44c4c6e87d79a, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:57.350217 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f593bc7-bdaf-4981-a589-94630a7005e9 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:57.368170 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 153/313] 10.222.0.42 () {58 vars in 1334 bytes} [Thu Sep 8 00:59:57 2022] PUT /identity/v3/system/users/36d836f65f724326a9a44c4c6e87d79a/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 00:59:57.402587 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd162d4-151d-4f2c-88f3-62f7f4f1d541 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:57.446575 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58fd4c0-353f-49c8-bdd5-f96f5cdc7fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:57.447270 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58fd4c0-353f-49c8-bdd5-f96f5cdc7fa9 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:57.447556 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58fd4c0-353f-49c8-bdd5-f96f5cdc7fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:57.447838 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58fd4c0-353f-49c8-bdd5-f96f5cdc7fa9 None admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:57.448634 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58fd4c0-353f-49c8-bdd5-f96f5cdc7fa9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:57.448943 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58fd4c0-353f-49c8-bdd5-f96f5cdc7fa9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:57.450520 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58fd4c0-353f-49c8-bdd5-f96f5cdc7fa9 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:57.461101 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 161/314] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 8 00:59:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 00:59:57.467696 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f593bc7-bdaf-4981-a589-94630a7005e9 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:57.584059 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f593bc7-bdaf-4981-a589-94630a7005e9 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:57.619656 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0218aefe-ed2f-4ba6-931f-c99a802f43c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:57.620332 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0218aefe-ed2f-4ba6-931f-c99a802f43c0 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:57.620615 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0218aefe-ed2f-4ba6-931f-c99a802f43c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:57.620887 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0218aefe-ed2f-4ba6-931f-c99a802f43c0 None admin] PATH_INFO: `/v3/users/ca8b7995da7e4dfaa839b2c9579a9577` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:57.622091 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0218aefe-ed2f-4ba6-931f-c99a802f43c0 None admin] RBAC: Authorizing `identity:get_user(user_id=ca8b7995da7e4dfaa839b2c9579a9577)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:57.625834 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0218aefe-ed2f-4ba6-931f-c99a802f43c0 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:57.625966 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 154/315] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 8 00:59:57 2022] GET /identity/v3/users/ca8b7995da7e4dfaa839b2c9579a9577 => generated 297 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:57.632348 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58fd4c0-353f-49c8-bdd5-f96f5cdc7fa9 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:57.647612 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:57.741999 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58fd4c0-353f-49c8-bdd5-f96f5cdc7fa9 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:57.777482 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e405cabb-08a0-4570-be41-a01f600776f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:57.778098 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405cabb-08a0-4570-be41-a01f600776f0 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:57.778442 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405cabb-08a0-4570-be41-a01f600776f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:57.779293 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405cabb-08a0-4570-be41-a01f600776f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:57.849128 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e405cabb-08a0-4570-be41-a01f600776f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:57.850544 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e405cabb-08a0-4570-be41-a01f600776f0 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:57.862523 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:57.954902 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 162/316] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 00:59:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 00:59:57.960540 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0218aefe-ed2f-4ba6-931f-c99a802f43c0 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:57.982302 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:58.072491 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0218aefe-ed2f-4ba6-931f-c99a802f43c0 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:58.108814 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da5652-5dce-4bd5-97ce-3e17d81d2af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:58.109419 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da5652-5dce-4bd5-97ce-3e17d81d2af8 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:58.109707 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da5652-5dce-4bd5-97ce-3e17d81d2af8 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:58.109981 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da5652-5dce-4bd5-97ce-3e17d81d2af8 None admin] PATH_INFO: `/v3/projects/0f19f058a73f49819b30c33d7fcfc52f` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:58.111117 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da5652-5dce-4bd5-97ce-3e17d81d2af8 None admin] RBAC: Authorizing `identity:get_project(project_id=0f19f058a73f49819b30c33d7fcfc52f)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:58.112614 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da5652-5dce-4bd5-97ce-3e17d81d2af8 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:58.114536 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 155/317] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 00:59:57 2022] GET /identity/v3/projects/0f19f058a73f49819b30c33d7fcfc52f => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:58.120371 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e405cabb-08a0-4570-be41-a01f600776f0 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:58.223545 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e405cabb-08a0-4570-be41-a01f600776f0 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:58.234463 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:58.259632 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1517d4e-ee35-403e-9a19-a9bd43c3f43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:58.260278 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1517d4e-ee35-403e-9a19-a9bd43c3f43b None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:58.260813 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1517d4e-ee35-403e-9a19-a9bd43c3f43b None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:58.261089 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1517d4e-ee35-403e-9a19-a9bd43c3f43b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:58.261819 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1517d4e-ee35-403e-9a19-a9bd43c3f43b None admin] RBAC: Adding query filter params (role.id=653150c294214a0da3bdd8d00aacd3aa, scope.project.id=0f19f058a73f49819b30c33d7fcfc52f, user.id=ca8b7995da7e4dfaa839b2c9579a9577) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 00:59:58.262128 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1517d4e-ee35-403e-9a19-a9bd43c3f43b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:58.264303 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1517d4e-ee35-403e-9a19-a9bd43c3f43b None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:58.271137 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 163/318] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 8 00:59:58 2022] GET /identity/v3/role_assignments?user.id=ca8b7995da7e4dfaa839b2c9579a9577&scope.project.id=0f19f058a73f49819b30c33d7fcfc52f&role.id=653150c294214a0da3bdd8d00aacd3aa => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:58.978529 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b047ed8-d5d3-4cd4-a50b-037a2a68702b None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:58.979022 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b047ed8-d5d3-4cd4-a50b-037a2a68702b None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:58.979022 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b047ed8-d5d3-4cd4-a50b-037a2a68702b None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:58.980504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 156/319] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 00:59:58.985811 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbaf12f-00f5-47d6-b078-9e22d5c6c709 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:58.986201 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbaf12f-00f5-47d6-b078-9e22d5c6c709 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:58.986360 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbaf12f-00f5-47d6-b078-9e22d5c6c709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:59.005425 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-acbaf12f-00f5-47d6-b078-9e22d5c6c709 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:59.006621 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-acbaf12f-00f5-47d6-b078-9e22d5c6c709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:59.090184 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 164/320] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 00:59:59.359284 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b047ed8-d5d3-4cd4-a50b-037a2a68702b None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:59.379363 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e20c1-e6ff-4734-b385-27a4115d7d09 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:59.379653 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e20c1-e6ff-4734-b385-27a4115d7d09 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:59.379922 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e20c1-e6ff-4734-b385-27a4115d7d09 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:59.381290 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 165/321] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 00:59:59.386218 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da7dc2-8080-4ef9-bf8c-7886a5de5b6b None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:59.386500 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da7dc2-8080-4ef9-bf8c-7886a5de5b6b None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:59.386773 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da7dc2-8080-4ef9-bf8c-7886a5de5b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:59.405642 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-22da7dc2-8080-4ef9-bf8c-7886a5de5b6b None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:59.406822 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-22da7dc2-8080-4ef9-bf8c-7886a5de5b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:59.459943 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:59.478656 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b047ed8-d5d3-4cd4-a50b-037a2a68702b None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:59.484385 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 166/322] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 00:59:59.513499 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b44f083-adc9-41ed-8bde-28b2d15e6893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:59.514381 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b44f083-adc9-41ed-8bde-28b2d15e6893 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:59.514803 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b44f083-adc9-41ed-8bde-28b2d15e6893 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:59.515228 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b44f083-adc9-41ed-8bde-28b2d15e6893 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:59.516491 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b44f083-adc9-41ed-8bde-28b2d15e6893 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:59.518087 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b44f083-adc9-41ed-8bde-28b2d15e6893 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:59.519807 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 157/323] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 00:59:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 00:59:59.525435 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22da7dc2-8080-4ef9-bf8c-7886a5de5b6b None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:59.553966 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85fbf24-800a-4a75-856d-b8ac87a47125 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:59.554283 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85fbf24-800a-4a75-856d-b8ac87a47125 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:59.554588 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85fbf24-800a-4a75-856d-b8ac87a47125 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:59.555928 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 158/324] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 00:59:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 00:59:59.561339 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b1980b-3892-40aa-affd-e3537f27b726 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:59.561624 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b1980b-3892-40aa-affd-e3537f27b726 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:59.561899 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b1980b-3892-40aa-affd-e3537f27b726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:59.563244 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:59.580574 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-80b1980b-3892-40aa-affd-e3537f27b726 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 00:59:59.581567 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-80b1980b-3892-40aa-affd-e3537f27b726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 00:59:59.635170 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22da7dc2-8080-4ef9-bf8c-7886a5de5b6b None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:59.657876 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 159/325] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 00:59:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 00:59:59.672432 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d69036-a2af-4f90-ae18-aa6029074e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:59.673166 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d69036-a2af-4f90-ae18-aa6029074e5d None admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:59.673456 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d69036-a2af-4f90-ae18-aa6029074e5d None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:59.673779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d69036-a2af-4f90-ae18-aa6029074e5d None admin] PATH_INFO: `/v3/groups` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:59.674422 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d69036-a2af-4f90-ae18-aa6029074e5d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 00:59:59.675914 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d69036-a2af-4f90-ae18-aa6029074e5d None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 00:59:59.692411 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 167/326] 10.222.0.42 () {60 vars in 1134 bytes} [Thu Sep 8 00:59:59 2022] POST /identity/v3/groups => generated 226 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 00:59:59.755288 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b1980b-3892-40aa-affd-e3537f27b726 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:59.851193 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 00:59:59.873129 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b1980b-3892-40aa-affd-e3537f27b726 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 00:59:59.908888 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797b4292-c6ce-4e63-9fba-1536cd338734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 00:59:59.909452 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b4292-c6ce-4e63-9fba-1536cd338734 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 00:59:59.909733 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b4292-c6ce-4e63-9fba-1536cd338734 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 00:59:59.910008 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b4292-c6ce-4e63-9fba-1536cd338734 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 00:59:59.921343 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797b4292-c6ce-4e63-9fba-1536cd338734 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 00:59:59.926197 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-797b4292-c6ce-4e63-9fba-1536cd338734 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 00:59:59.926197 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 00:59:59.926197 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 00:59:59.926197 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 00:59:59.926197 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 00:59:59.926197 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 00:59:59.926197 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 00:59:59.926197 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 00:59:59.926197 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 00:59:59.926197 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 00:59:59.926197 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 00:59:59.926197 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 00:59:59.926197 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 00:59:59.926197 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 00:59:59.926197 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 00:59:59.926197 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 00:59:59.927070 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 00:59:59.927070 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 00:59:59.927070 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 00:59:59.927070 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 00:59:59.927070 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 00:59:59.927070 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 00:59:59.927070 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 00:59:59.927070 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 00:59:59.927070 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 00:59:59.927070 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 00:59:59.927070 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 00:59:59.927070 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 00:59:59.927070 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 00:59:59.927070 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 00:59:59.927070 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 00:59:59.927070 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 00:59:59.927875 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 00:59:59.927875 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 00:59:59.927875 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 00:59:59.927875 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 00:59:59.927875 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 00:59:59.927875 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 00:59:59.927875 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 00:59:59.927875 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 00:59:59.927875 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 00:59:59.927875 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 00:59:59.927875 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 00:59:59.927875 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 00:59:59.927875 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 160/327] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 00:59:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 00:59:59.933276 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d69036-a2af-4f90-ae18-aa6029074e5d None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 00:59:59.937161 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797b4292-c6ce-4e63-9fba-1536cd338734 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:00.056382 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:00.066647 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d69036-a2af-4f90-ae18-aa6029074e5d None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:00.071326 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797b4292-c6ce-4e63-9fba-1536cd338734 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:00.113458 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab1fd65-3d5a-4bd9-96fe-ac799e0b41bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:00.114513 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab1fd65-3d5a-4bd9-96fe-ac799e0b41bd None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:00.115059 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab1fd65-3d5a-4bd9-96fe-ac799e0b41bd None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:00.115516 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab1fd65-3d5a-4bd9-96fe-ac799e0b41bd None admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:00.115661 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15a2719-f7bb-4658-b85a-7640ad62ce2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:00.116225 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab1fd65-3d5a-4bd9-96fe-ac799e0b41bd None admin] RBAC: Adding query filter params (name=reader) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:00.116324 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a2719-f7bb-4658-b85a-7640ad62ce2f None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:00.116532 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab1fd65-3d5a-4bd9-96fe-ac799e0b41bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:00.116625 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a2719-f7bb-4658-b85a-7640ad62ce2f None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:00.116890 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a2719-f7bb-4658-b85a-7640ad62ce2f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:00.118127 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab1fd65-3d5a-4bd9-96fe-ac799e0b41bd None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:00.129179 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 168/328] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 8 00:59:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:00:00.130302 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15a2719-f7bb-4658-b85a-7640ad62ce2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:00:00.135352 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-f15a2719-f7bb-4658-b85a-7640ad62ce2f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 01:00:00.135352 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:00.135352 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:00.135352 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:00.135352 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:00.135352 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:00.135352 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:00.135352 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:00.135352 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:00.135352 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:00.135352 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:00.135352 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:00.135352 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:00:00.135352 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:00:00.135352 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:00:00.135352 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:00:00.136531 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:00:00.136531 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:00:00.136531 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:00:00.136531 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:00:00.136531 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:00.136531 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:00.136531 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:00.136531 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:00.136531 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:00.136531 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:00.136531 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:00.136531 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:00.136531 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:00.136531 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:00.136531 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:00.136531 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:00.137318 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:00.137318 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:00.137318 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:00.137318 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:00.137318 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:00:00.137318 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:00:00.137318 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:00:00.137318 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:00:00.137318 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:00:00.137318 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:00:00.137318 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 01:00:00.137318 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:00.138605 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab1fd65-3d5a-4bd9-96fe-ac799e0b41bd None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:00.141509 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 161/329] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 00:59:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 01:00:00.148106 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15a2719-f7bb-4658-b85a-7640ad62ce2f None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:00.156976 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:00.256254 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab1fd65-3d5a-4bd9-96fe-ac799e0b41bd None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:00.284510 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15a2719-f7bb-4658-b85a-7640ad62ce2f None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:00.299184 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b815f413-e3f9-4446-8b6e-86e36ba981d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:00.300073 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b815f413-e3f9-4446-8b6e-86e36ba981d0 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:00.300073 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b815f413-e3f9-4446-8b6e-86e36ba981d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:00.300378 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b815f413-e3f9-4446-8b6e-86e36ba981d0 None admin] PATH_INFO: `/v3/users/640b55004f3f4278b286d4f1e6ef2132` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:00.301513 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b815f413-e3f9-4446-8b6e-86e36ba981d0 None admin] RBAC: Authorizing `identity:get_user(user_id=640b55004f3f4278b286d4f1e6ef2132)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:00.302967 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b815f413-e3f9-4446-8b6e-86e36ba981d0 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:00.304891 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 169/330] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 8 01:00:00 2022] GET /identity/v3/users/640b55004f3f4278b286d4f1e6ef2132 => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:00.310910 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b815f413-e3f9-4446-8b6e-86e36ba981d0 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:00.328382 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9ac55b-081e-410b-8133-f9a913226247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:00.329262 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ac55b-081e-410b-8133-f9a913226247 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:00.330051 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ac55b-081e-410b-8133-f9a913226247 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:00.330330 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ac55b-081e-410b-8133-f9a913226247 None admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:00.331306 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9ac55b-081e-410b-8133-f9a913226247 None admin] RBAC: Adding query filter params (name=member) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:00.331606 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9ac55b-081e-410b-8133-f9a913226247 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:00.333306 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9ac55b-081e-410b-8133-f9a913226247 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:00.344182 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:00.345868 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 162/331] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 8 01:00:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:00:00.353694 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9ac55b-081e-410b-8133-f9a913226247 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:00.437980 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b815f413-e3f9-4446-8b6e-86e36ba981d0 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:00.472541 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:00.484922 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e60c41d-d456-40dc-a82a-f643e89d37d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:00.485636 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60c41d-d456-40dc-a82a-f643e89d37d7 None admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:00.485921 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60c41d-d456-40dc-a82a-f643e89d37d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:00.486194 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60c41d-d456-40dc-a82a-f643e89d37d7 None admin] PATH_INFO: `/v3/system/users/640b55004f3f4278b286d4f1e6ef2132/roles/653150c294214a0da3bdd8d00aacd3aa` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:00.487630 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e60c41d-d456-40dc-a82a-f643e89d37d7 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=640b55004f3f4278b286d4f1e6ef2132, role_id=653150c294214a0da3bdd8d00aacd3aa)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:00.489057 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e60c41d-d456-40dc-a82a-f643e89d37d7 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:00.499397 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9ac55b-081e-410b-8133-f9a913226247 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:00.502086 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 170/332] 10.222.0.42 () {58 vars in 1334 bytes} [Thu Sep 8 01:00:00 2022] PUT /identity/v3/system/users/640b55004f3f4278b286d4f1e6ef2132/roles/653150c294214a0da3bdd8d00aacd3aa => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:00:00.548660 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f989bc-5b68-47cc-9d69-39fb814be471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:00.549654 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f989bc-5b68-47cc-9d69-39fb814be471 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:00.550116 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f989bc-5b68-47cc-9d69-39fb814be471 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:00.550590 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f989bc-5b68-47cc-9d69-39fb814be471 None admin] PATH_INFO: `/v3/users/36d836f65f724326a9a44c4c6e87d79a` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:00.551873 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f989bc-5b68-47cc-9d69-39fb814be471 None admin] RBAC: Authorizing `identity:get_user(user_id=36d836f65f724326a9a44c4c6e87d79a)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:00.553681 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f989bc-5b68-47cc-9d69-39fb814be471 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:00.560254 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 163/333] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 8 01:00:00 2022] GET /identity/v3/users/36d836f65f724326a9a44c4c6e87d79a => generated 293 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:00:00.561632 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e60c41d-d456-40dc-a82a-f643e89d37d7 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:00.674023 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:00.688095 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e60c41d-d456-40dc-a82a-f643e89d37d7 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:00.726765 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc9bd72-132f-4779-a120-1e1192d9b749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:00.727407 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9bd72-132f-4779-a120-1e1192d9b749 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:00.727689 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9bd72-132f-4779-a120-1e1192d9b749 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:00.728005 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9bd72-132f-4779-a120-1e1192d9b749 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:00.728859 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc9bd72-132f-4779-a120-1e1192d9b749 None admin] RBAC: Adding query filter params (role.id=92aa6d6824404d10b4a3be8b4a6bc317, user.id=36d836f65f724326a9a44c4c6e87d79a, scope.system=all) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:00.729176 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc9bd72-132f-4779-a120-1e1192d9b749 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:00.730638 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc9bd72-132f-4779-a120-1e1192d9b749 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:00.746462 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 171/334] 10.222.0.42 () {56 vars in 1308 bytes} [Thu Sep 8 01:00:00 2022] GET /identity/v3/role_assignments?user.id=36d836f65f724326a9a44c4c6e87d79a&scope.system=all&role.id=92aa6d6824404d10b4a3be8b4a6bc317 => generated 510 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:02.491066 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ec0cf-6bf6-4e5f-a51c-2c42cf70843a None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:02.492122 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ec0cf-6bf6-4e5f-a51c-2c42cf70843a None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:02.492632 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ec0cf-6bf6-4e5f-a51c-2c42cf70843a None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:02.494485 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 164/335] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:00:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:00:02.502263 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e0375-7464-49bc-8cf4-124605ef0c2d None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:02.503137 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e0375-7464-49bc-8cf4-124605ef0c2d None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:02.503609 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e0375-7464-49bc-8cf4-124605ef0c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:02.529847 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-168e0375-7464-49bc-8cf4-124605ef0c2d None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:00:02.531399 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-168e0375-7464-49bc-8cf4-124605ef0c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:00:02.614289 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 172/336] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:00:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 01:00:02.730157 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db78bf6-49a7-43a8-a573-862ea4aa81e5 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:02.730812 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db78bf6-49a7-43a8-a573-862ea4aa81e5 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:02.731287 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db78bf6-49a7-43a8-a573-862ea4aa81e5 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:02.733157 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 165/337] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:00:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:00:02.739148 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d75afa-d8f9-4ded-b219-03a3b7e57b51 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:02.739679 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d75afa-d8f9-4ded-b219-03a3b7e57b51 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:02.740209 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d75afa-d8f9-4ded-b219-03a3b7e57b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:02.765111 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-62d75afa-d8f9-4ded-b219-03a3b7e57b51 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:00:02.766823 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-62d75afa-d8f9-4ded-b219-03a3b7e57b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:00:02.845741 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 173/338] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:00:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 01:00:03.039321 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db78bf6-49a7-43a8-a573-862ea4aa81e5 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:03.170641 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db78bf6-49a7-43a8-a573-862ea4aa81e5 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:03.210016 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c8e9b7-92fd-4f01-887e-e409b7bbf4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:03.211043 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8e9b7-92fd-4f01-887e-e409b7bbf4be None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:03.211574 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8e9b7-92fd-4f01-887e-e409b7bbf4be None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:03.212067 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8e9b7-92fd-4f01-887e-e409b7bbf4be None admin] PATH_INFO: `/v3/roles/member` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:03.217782 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:03.227418 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c8e9b7-92fd-4f01-887e-e409b7bbf4be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:00:03.235483 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-94c8e9b7-92fd-4f01-887e-e409b7bbf4be None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 01:00:03.235483 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:03.235483 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:03.235483 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:03.235483 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:03.235483 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:03.235483 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:03.235483 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:03.235483 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:03.235483 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:03.235483 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:03.235483 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:03.235483 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:00:03.235483 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:00:03.235483 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:00:03.235483 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:00:03.236425 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:00:03.236425 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:00:03.236425 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:00:03.236425 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:00:03.236425 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:03.236425 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:03.236425 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:03.236425 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:03.236425 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:03.236425 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:03.236425 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:03.236425 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:03.236425 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:03.236425 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:03.236425 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:03.236425 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:03.237200 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:03.237200 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:03.237200 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:03.237200 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:03.237200 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:00:03.237200 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:00:03.237200 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:00:03.237200 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:00:03.237200 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:00:03.237200 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:00:03.237200 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 01:00:03.237200 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:03.237200 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 166/339] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 01:00:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:03.242931 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d75afa-d8f9-4ded-b219-03a3b7e57b51 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:03.252723 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:03.387946 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d75afa-d8f9-4ded-b219-03a3b7e57b51 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:03.427068 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d0fa3e-6d58-4db3-9dd0-d5a4becba850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:03.427682 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d0fa3e-6d58-4db3-9dd0-d5a4becba850 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:03.427956 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d0fa3e-6d58-4db3-9dd0-d5a4becba850 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:03.428244 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d0fa3e-6d58-4db3-9dd0-d5a4becba850 None admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:03.428916 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d0fa3e-6d58-4db3-9dd0-d5a4becba850 None admin] RBAC: Adding query filter params (name=member) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:03.429216 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d0fa3e-6d58-4db3-9dd0-d5a4becba850 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:03.430600 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d0fa3e-6d58-4db3-9dd0-d5a4becba850 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:03.439795 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 174/340] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 8 01:00:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:03.446233 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c8e9b7-92fd-4f01-887e-e409b7bbf4be None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:03.482227 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d0fa3e-6d58-4db3-9dd0-d5a4becba850 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:03.586873 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c8e9b7-92fd-4f01-887e-e409b7bbf4be None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:03.600555 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:03.635440 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade68a8-691f-469f-99c7-1897c89a426d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:03.636627 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade68a8-691f-469f-99c7-1897c89a426d None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:03.637107 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade68a8-691f-469f-99c7-1897c89a426d None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:03.638141 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade68a8-691f-469f-99c7-1897c89a426d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:03.652079 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d0fa3e-6d58-4db3-9dd0-d5a4becba850 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:03.721505 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c916586c-23c7-49ba-b078-0e2273755fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:03.723374 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916586c-23c7-49ba-b078-0e2273755fef None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:03.724653 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916586c-23c7-49ba-b078-0e2273755fef None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:03.725297 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916586c-23c7-49ba-b078-0e2273755fef None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:03.745096 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c916586c-23c7-49ba-b078-0e2273755fef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:00:03.753224 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-c916586c-23c7-49ba-b078-0e2273755fef None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 01:00:03.753224 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:03.753224 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:03.753224 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:03.753224 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:03.753224 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:03.753224 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:03.753224 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:03.753224 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:03.753224 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:03.753224 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:03.753224 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:03.753224 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:00:03.753224 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:00:03.753224 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:00:03.753224 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:00:03.754104 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:00:03.754104 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:00:03.754104 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:00:03.754104 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:00:03.754104 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:03.754104 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:03.754104 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:03.754104 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:03.754104 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:03.754104 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:03.754104 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:03.754104 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:03.754104 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:03.754104 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:03.754104 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:03.754104 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:03.754893 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:03.754893 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:03.754893 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:03.754893 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:03.754893 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:00:03.754893 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:00:03.754893 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:00:03.754893 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:00:03.754893 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:00:03.754893 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:00:03.754893 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 01:00:03.754893 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:03.758398 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:03.766216 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 175/341] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 8 01:00:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:03.766216 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463fd14a-cdc8-480d-9ac2-7f079352d9e6 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:03.766216 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463fd14a-cdc8-480d-9ac2-7f079352d9e6 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:03.766216 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463fd14a-cdc8-480d-9ac2-7f079352d9e6 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:03.766216 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 176/342] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:00:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:00:03.775664 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc28e11-47ad-4718-b690-eae217868643 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:03.776707 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc28e11-47ad-4718-b690-eae217868643 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:03.777132 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc28e11-47ad-4718-b690-eae217868643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:03.779001 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cade68a8-691f-469f-99c7-1897c89a426d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:03.780654 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cade68a8-691f-469f-99c7-1897c89a426d None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:03.811331 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-bdc28e11-47ad-4718-b690-eae217868643 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:00:03.812978 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-bdc28e11-47ad-4718-b690-eae217868643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:00:03.920040 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 177/343] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:00:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 01:00:03.920793 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc28e11-47ad-4718-b690-eae217868643 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:03.977760 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 167/344] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 01:00:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 01:00:03.984690 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade68a8-691f-469f-99c7-1897c89a426d None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:04.048531 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:04.134390 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc28e11-47ad-4718-b690-eae217868643 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:04.165901 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:04.178744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade68a8-691f-469f-99c7-1897c89a426d None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:04.190791 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f166d6d-f534-4741-819e-1a3b6fa080d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:04.192004 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f166d6d-f534-4741-819e-1a3b6fa080d4 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:04.192527 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f166d6d-f534-4741-819e-1a3b6fa080d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:04.193245 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f166d6d-f534-4741-819e-1a3b6fa080d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:04.194128 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f166d6d-f534-4741-819e-1a3b6fa080d4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:04.194625 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f166d6d-f534-4741-819e-1a3b6fa080d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:04.196332 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f166d6d-f534-4741-819e-1a3b6fa080d4 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:04.206314 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 178/345] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 01:00:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:04.207912 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c8064-f08d-4aac-bf43-0793bdebba98 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:04.212480 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c8064-f08d-4aac-bf43-0793bdebba98 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:04.213358 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c8064-f08d-4aac-bf43-0793bdebba98 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:04.214044 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 179/346] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:00:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:00:04.215094 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8015c04b-6ad5-4b78-bb0d-810169f8fffe None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:04.216674 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8015c04b-6ad5-4b78-bb0d-810169f8fffe None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:04.216950 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8015c04b-6ad5-4b78-bb0d-810169f8fffe None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:04.218490 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 180/347] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:00:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:00:04.219035 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8015c04b-6ad5-4b78-bb0d-810169f8fffe None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:04.229074 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abda89d-9932-4840-86e2-ba7dd9ac942f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:04.229706 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abda89d-9932-4840-86e2-ba7dd9ac942f None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:04.229978 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abda89d-9932-4840-86e2-ba7dd9ac942f None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:04.230313 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abda89d-9932-4840-86e2-ba7dd9ac942f None admin] PATH_INFO: `/v3/projects/41f15a3cf54f4397920b0e8f758539ea` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:04.232717 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abda89d-9932-4840-86e2-ba7dd9ac942f None admin] RBAC: Authorizing `identity:get_project(project_id=41f15a3cf54f4397920b0e8f758539ea)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:04.233202 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abda89d-9932-4840-86e2-ba7dd9ac942f None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:04.234988 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 168/348] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 01:00:03 2022] GET /identity/v3/projects/41f15a3cf54f4397920b0e8f758539ea => generated 298 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:00:04.235993 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1605a75b-60b6-46a1-83dd-583e7d575a7f None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:04.239373 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1605a75b-60b6-46a1-83dd-583e7d575a7f None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:04.239668 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1605a75b-60b6-46a1-83dd-583e7d575a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:04.259705 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-1605a75b-60b6-46a1-83dd-583e7d575a7f None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:00:04.262106 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-1605a75b-60b6-46a1-83dd-583e7d575a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:00:04.337366 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 169/349] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:00:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 01:00:04.338620 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb0719-56e8-40ec-8d00-ff538997afc9 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:04.338901 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb0719-56e8-40ec-8d00-ff538997afc9 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:04.339169 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb0719-56e8-40ec-8d00-ff538997afc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:04.341278 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8015c04b-6ad5-4b78-bb0d-810169f8fffe None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:04.358403 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-79eb0719-56e8-40ec-8d00-ff538997afc9 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:00:04.359584 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-79eb0719-56e8-40ec-8d00-ff538997afc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:00:04.364626 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:04.388641 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91390d2-cbf7-4083-be05-6c64b944e106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:04.389291 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91390d2-cbf7-4083-be05-6c64b944e106 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:04.389581 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91390d2-cbf7-4083-be05-6c64b944e106 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:04.389840 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91390d2-cbf7-4083-be05-6c64b944e106 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:04.465560 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 170/350] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:00:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 01:00:04.467179 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb0719-56e8-40ec-8d00-ff538997afc9 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:04.475512 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91390d2-cbf7-4083-be05-6c64b944e106 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:04.477728 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91390d2-cbf7-4083-be05-6c64b944e106 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:04.503240 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:04.593650 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb0719-56e8-40ec-8d00-ff538997afc9 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:04.610704 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 181/351] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 01:00:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 01:00:04.621617 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91390d2-cbf7-4083-be05-6c64b944e106 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:04.642336 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5b237d-8711-4699-8ef5-c1c83b913c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:04.643395 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b237d-8711-4699-8ef5-c1c83b913c04 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:04.643671 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b237d-8711-4699-8ef5-c1c83b913c04 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:04.643937 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b237d-8711-4699-8ef5-c1c83b913c04 None admin] PATH_INFO: `/v3/groups/0115efc9f6184a23bcf4d23ff19c2115` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:04.655552 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:04.660188 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5b237d-8711-4699-8ef5-c1c83b913c04 None admin] RBAC: Authorizing `identity:get_group(group_id=0115efc9f6184a23bcf4d23ff19c2115)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:04.660188 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5b237d-8711-4699-8ef5-c1c83b913c04 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:04.660188 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 171/352] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Sep 8 01:00:04 2022] GET /identity/v3/groups/0115efc9f6184a23bcf4d23ff19c2115 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:04.660740 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5b237d-8711-4699-8ef5-c1c83b913c04 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:04.797158 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91390d2-cbf7-4083-be05-6c64b944e106 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:04.805520 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:04.828381 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5b237d-8711-4699-8ef5-c1c83b913c04 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:04.848004 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54491630-1e13-4ba2-b074-9e197605ba1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:04.849101 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54491630-1e13-4ba2-b074-9e197605ba1a None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:04.849609 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54491630-1e13-4ba2-b074-9e197605ba1a None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:04.850116 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54491630-1e13-4ba2-b074-9e197605ba1a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:04.866140 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54491630-1e13-4ba2-b074-9e197605ba1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:00:04.870247 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-54491630-1e13-4ba2-b074-9e197605ba1a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 01:00:04.870247 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:04.870247 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:04.870247 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:04.870247 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:04.870247 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:04.870247 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:04.870247 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:04.870247 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:04.870247 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:04.870247 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:04.870247 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:04.870247 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:00:04.870247 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:00:04.870247 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:00:04.870247 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:00:04.871151 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:00:04.871151 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:00:04.871151 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:00:04.871151 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:00:04.871151 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:04.871151 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:04.871151 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:04.871151 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:04.871151 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:04.871151 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:04.871151 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:04.871151 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:04.871151 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:04.871151 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:04.871151 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:04.871151 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:04.872020 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:04.872020 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:04.872020 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:04.872020 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:04.872020 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:00:04.872020 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:00:04.872020 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:00:04.872020 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:00:04.872020 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:00:04.872020 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:00:04.872020 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 01:00:04.872020 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:04.872020 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 182/353] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 01:00:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:04.872762 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54491630-1e13-4ba2-b074-9e197605ba1a None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:04.878890 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd861e3-63a5-4817-a620-2baf3b518d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:04.879539 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd861e3-63a5-4817-a620-2baf3b518d88 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:04.879819 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd861e3-63a5-4817-a620-2baf3b518d88 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:04.880091 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd861e3-63a5-4817-a620-2baf3b518d88 None admin] PATH_INFO: `/v3/projects/41f15a3cf54f4397920b0e8f758539ea` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:04.881256 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd861e3-63a5-4817-a620-2baf3b518d88 None admin] RBAC: Authorizing `identity:get_project(project_id=41f15a3cf54f4397920b0e8f758539ea)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:04.882733 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd861e3-63a5-4817-a620-2baf3b518d88 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:04.884446 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 172/354] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 01:00:04 2022] GET /identity/v3/projects/41f15a3cf54f4397920b0e8f758539ea => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:04.885200 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd861e3-63a5-4817-a620-2baf3b518d88 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:04.980108 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:05.005470 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54491630-1e13-4ba2-b074-9e197605ba1a None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:05.028460 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd861e3-63a5-4817-a620-2baf3b518d88 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:05.057686 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9257192-6fda-4489-b2fa-35d3d429a044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:05.058230 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9257192-6fda-4489-b2fa-35d3d429a044 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:05.058510 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9257192-6fda-4489-b2fa-35d3d429a044 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:05.058785 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9257192-6fda-4489-b2fa-35d3d429a044 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:05.059588 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9257192-6fda-4489-b2fa-35d3d429a044 None admin] RBAC: Adding query filter params (group.id=0115efc9f6184a23bcf4d23ff19c2115, role.id=92aa6d6824404d10b4a3be8b4a6bc317, scope.project.id=41f15a3cf54f4397920b0e8f758539ea) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:05.059900 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9257192-6fda-4489-b2fa-35d3d429a044 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:05.061407 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9257192-6fda-4489-b2fa-35d3d429a044 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:05.067117 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 183/355] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Sep 8 01:00:04 2022] GET /identity/v3/role_assignments?group.id=0115efc9f6184a23bcf4d23ff19c2115&scope.project.id=41f15a3cf54f4397920b0e8f758539ea&role.id=92aa6d6824404d10b4a3be8b4a6bc317 => generated 260 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:05.067969 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9257192-6fda-4489-b2fa-35d3d429a044 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:05.071635 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d87919-9ed7-461b-b80c-286ec9eb1f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:05.076166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d87919-9ed7-461b-b80c-286ec9eb1f86 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:05.076166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d87919-9ed7-461b-b80c-286ec9eb1f86 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:05.076166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d87919-9ed7-461b-b80c-286ec9eb1f86 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:05.094785 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d87919-9ed7-461b-b80c-286ec9eb1f86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:00:05.102072 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-03d87919-9ed7-461b-b80c-286ec9eb1f86 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 01:00:05.102072 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:05.102072 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:05.102072 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:05.102072 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:05.102072 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:05.102072 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:05.102072 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:05.102072 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:05.102072 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:05.102072 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:05.102072 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:05.102072 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:00:05.102072 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:00:05.102072 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:00:05.102072 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:00:05.103026 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:00:05.103026 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:00:05.103026 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:00:05.103026 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:00:05.103026 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:05.103026 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:05.103026 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:05.103026 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:05.103026 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:05.103026 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:05.103026 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:05.103026 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:05.103026 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:05.103026 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:05.103026 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:05.103026 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:05.103811 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:05.103811 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:05.103811 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:05.103811 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:05.103811 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:00:05.103811 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:00:05.103811 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:00:05.103811 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:00:05.103811 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:00:05.103811 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:00:05.103811 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 01:00:05.103811 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:05.104599 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 173/356] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 01:00:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:05.106249 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d87919-9ed7-461b-b80c-286ec9eb1f86 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:05.126017 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:05.218354 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9257192-6fda-4489-b2fa-35d3d429a044 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:05.247963 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d87919-9ed7-461b-b80c-286ec9eb1f86 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:05.262374 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f1a550-fc38-4443-9db8-1da6b47e7cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:05.263111 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1a550-fc38-4443-9db8-1da6b47e7cc7 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:05.263392 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1a550-fc38-4443-9db8-1da6b47e7cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:05.263860 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1a550-fc38-4443-9db8-1da6b47e7cc7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:05.277693 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f1a550-fc38-4443-9db8-1da6b47e7cc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:00:05.286834 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-23f1a550-fc38-4443-9db8-1da6b47e7cc7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 01:00:05.286834 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:05.286834 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:05.286834 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:05.286834 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:05.286834 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:05.286834 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:05.286834 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:05.286834 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:05.286834 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:05.286834 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:05.286834 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:05.286834 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:00:05.286834 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:00:05.286834 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:00:05.286834 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:00:05.288022 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:00:05.288022 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:00:05.288022 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:00:05.288022 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:00:05.288022 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:05.288022 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:05.288022 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:05.288022 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:05.288022 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:05.288022 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:05.288022 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:05.288022 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:05.288022 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:05.288022 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:05.288022 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:05.288022 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:05.289496 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:05.289496 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:05.289496 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:05.289496 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:05.289496 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:00:05.289496 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:00:05.289496 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:00:05.289496 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:00:05.289496 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:00:05.289496 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:00:05.289496 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 01:00:05.289496 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:05.289496 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 184/357] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 8 01:00:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:05.290396 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f1a550-fc38-4443-9db8-1da6b47e7cc7 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:05.299150 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a1f1a4-f7c1-446c-bae6-4faa3b140f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:05.299837 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:05.300708 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a1f1a4-f7c1-446c-bae6-4faa3b140f98 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:05.300708 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a1f1a4-f7c1-446c-bae6-4faa3b140f98 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:05.302186 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a1f1a4-f7c1-446c-bae6-4faa3b140f98 None admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:05.303016 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a1f1a4-f7c1-446c-bae6-4faa3b140f98 None admin] RBAC: Adding query filter params (name=member) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:05.303327 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a1f1a4-f7c1-446c-bae6-4faa3b140f98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:05.304788 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a1f1a4-f7c1-446c-bae6-4faa3b140f98 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:05.317373 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 174/358] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 8 01:00:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:05.317864 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a388eb-4fa9-4f75-ae5c-9a1022f2168c None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:05.318148 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a388eb-4fa9-4f75-ae5c-9a1022f2168c None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:05.318421 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a388eb-4fa9-4f75-ae5c-9a1022f2168c None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:05.319621 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 175/359] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:00:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:00:05.320334 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a388eb-4fa9-4f75-ae5c-9a1022f2168c None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:05.428198 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:05.430494 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f1a550-fc38-4443-9db8-1da6b47e7cc7 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:05.460598 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a388eb-4fa9-4f75-ae5c-9a1022f2168c None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:05.474888 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933c4415-f9bd-430d-a50b-6799d5a21f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:05.475857 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c4415-f9bd-430d-a50b-6799d5a21f9e None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:05.476169 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c4415-f9bd-430d-a50b-6799d5a21f9e None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:05.476775 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c4415-f9bd-430d-a50b-6799d5a21f9e None admin] PATH_INFO: `/v3/groups/0115efc9f6184a23bcf4d23ff19c2115` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:05.478036 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933c4415-f9bd-430d-a50b-6799d5a21f9e None admin] RBAC: Authorizing `identity:get_group(group_id=0115efc9f6184a23bcf4d23ff19c2115)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:05.479479 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933c4415-f9bd-430d-a50b-6799d5a21f9e None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:05.482137 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 185/360] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Sep 8 01:00:05 2022] GET /identity/v3/groups/0115efc9f6184a23bcf4d23ff19c2115 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:05.487923 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933c4415-f9bd-430d-a50b-6799d5a21f9e None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:05.514884 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2239333f-24a3-417e-a91b-4b27cc85bc6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:05.515887 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2239333f-24a3-417e-a91b-4b27cc85bc6d None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:05.516361 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2239333f-24a3-417e-a91b-4b27cc85bc6d None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:05.516855 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2239333f-24a3-417e-a91b-4b27cc85bc6d None admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:05.518128 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2239333f-24a3-417e-a91b-4b27cc85bc6d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:05.518708 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2239333f-24a3-417e-a91b-4b27cc85bc6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:05.520296 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2239333f-24a3-417e-a91b-4b27cc85bc6d None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:05.531676 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 176/361] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 8 01:00:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:05.532478 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2239333f-24a3-417e-a91b-4b27cc85bc6d None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:05.589967 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:05.628126 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933c4415-f9bd-430d-a50b-6799d5a21f9e None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:05.666460 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2239333f-24a3-417e-a91b-4b27cc85bc6d None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:05.680643 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94752ad0-daea-4020-bcc2-80ebf735e008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:05.681263 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94752ad0-daea-4020-bcc2-80ebf735e008 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:05.681542 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94752ad0-daea-4020-bcc2-80ebf735e008 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:05.681814 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94752ad0-daea-4020-bcc2-80ebf735e008 None admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:05.682488 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94752ad0-daea-4020-bcc2-80ebf735e008 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:05.682786 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94752ad0-daea-4020-bcc2-80ebf735e008 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:05.685361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94752ad0-daea-4020-bcc2-80ebf735e008 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:05.695852 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 186/362] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 01:00:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:05.701885 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c13f59-5317-471d-ba63-4de5f7826e88 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:05.702173 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c13f59-5317-471d-ba63-4de5f7826e88 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:05.702445 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c13f59-5317-471d-ba63-4de5f7826e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:05.708897 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee6d6f5-2b62-4ecf-82e0-caadd5eda60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:05.709623 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee6d6f5-2b62-4ecf-82e0-caadd5eda60e None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:05.710002 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee6d6f5-2b62-4ecf-82e0-caadd5eda60e None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:05.710373 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee6d6f5-2b62-4ecf-82e0-caadd5eda60e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:05.723169 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:05.727743 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-f0c13f59-5317-471d-ba63-4de5f7826e88 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:00:05.729041 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-f0c13f59-5317-471d-ba63-4de5f7826e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:00:05.805802 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee6d6f5-2b62-4ecf-82e0-caadd5eda60e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:05.807421 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee6d6f5-2b62-4ecf-82e0-caadd5eda60e None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:05.824126 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 187/363] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:00:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 01:00:05.825679 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c13f59-5317-471d-ba63-4de5f7826e88 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:05.909547 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:05.937814 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 177/364] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 01:00:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 01:00:05.939287 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee6d6f5-2b62-4ecf-82e0-caadd5eda60e None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:05.957418 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c13f59-5317-471d-ba63-4de5f7826e88 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:05.988781 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:05.997871 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3163dfe5-44dd-4237-823e-e0cd917804f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:05.998497 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3163dfe5-44dd-4237-823e-e0cd917804f8 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:05.998774 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3163dfe5-44dd-4237-823e-e0cd917804f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:05.999045 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3163dfe5-44dd-4237-823e-e0cd917804f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:06.000263 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3163dfe5-44dd-4237-823e-e0cd917804f8 None admin] RBAC: Adding query filter params (group.id=0115efc9f6184a23bcf4d23ff19c2115, role.id=069fe3e033664493b3def844e3735bc9, scope.project.id=41f15a3cf54f4397920b0e8f758539ea) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:06.000578 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3163dfe5-44dd-4237-823e-e0cd917804f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:06.002381 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3163dfe5-44dd-4237-823e-e0cd917804f8 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:06.009032 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 188/365] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Sep 8 01:00:05 2022] GET /identity/v3/role_assignments?group.id=0115efc9f6184a23bcf4d23ff19c2115&scope.project.id=41f15a3cf54f4397920b0e8f758539ea&role.id=069fe3e033664493b3def844e3735bc9 => generated 260 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:06.009918 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3163dfe5-44dd-4237-823e-e0cd917804f8 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:06.087468 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee6d6f5-2b62-4ecf-82e0-caadd5eda60e None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:06.132494 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e354138-9c90-430a-a9f0-8e053a246dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:06.133481 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3163dfe5-44dd-4237-823e-e0cd917804f8 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:06.133793 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e354138-9c90-430a-a9f0-8e053a246dde None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:06.134271 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e354138-9c90-430a-a9f0-8e053a246dde None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:06.134764 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e354138-9c90-430a-a9f0-8e053a246dde None admin] PATH_INFO: `/v3/users/640b55004f3f4278b286d4f1e6ef2132` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:06.136578 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e354138-9c90-430a-a9f0-8e053a246dde None admin] RBAC: Authorizing `identity:get_user(user_id=640b55004f3f4278b286d4f1e6ef2132)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:06.138426 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e354138-9c90-430a-a9f0-8e053a246dde None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:06.140456 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 178/366] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 8 01:00:05 2022] GET /identity/v3/users/640b55004f3f4278b286d4f1e6ef2132 => generated 293 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:06.141462 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e354138-9c90-430a-a9f0-8e053a246dde None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:06.172961 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:06.183696 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b0beca-6cca-4b7e-83bd-6fc79fabaea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:06.185166 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0beca-6cca-4b7e-83bd-6fc79fabaea4 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:06.185584 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0beca-6cca-4b7e-83bd-6fc79fabaea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:06.186006 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0beca-6cca-4b7e-83bd-6fc79fabaea4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:06.266804 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e354138-9c90-430a-a9f0-8e053a246dde None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:06.270556 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b0beca-6cca-4b7e-83bd-6fc79fabaea4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:06.272305 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b0beca-6cca-4b7e-83bd-6fc79fabaea4 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:06.276218 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:06.311671 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd80a5e7-6d30-4999-ad63-44a9a6413e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:06.312504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd80a5e7-6d30-4999-ad63-44a9a6413e82 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:06.312891 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd80a5e7-6d30-4999-ad63-44a9a6413e82 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:06.313268 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd80a5e7-6d30-4999-ad63-44a9a6413e82 None admin] PATH_INFO: `/v3/projects/0f19f058a73f49819b30c33d7fcfc52f` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:06.315608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd80a5e7-6d30-4999-ad63-44a9a6413e82 None admin] RBAC: Authorizing `identity:get_project(project_id=0f19f058a73f49819b30c33d7fcfc52f)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:06.317276 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd80a5e7-6d30-4999-ad63-44a9a6413e82 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:06.319880 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 179/367] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 01:00:06 2022] GET /identity/v3/projects/0f19f058a73f49819b30c33d7fcfc52f => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:06.320172 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd80a5e7-6d30-4999-ad63-44a9a6413e82 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:06.405533 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 189/368] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 01:00:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 01:00:06.406737 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b0beca-6cca-4b7e-83bd-6fc79fabaea4 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:06.444927 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:06.448411 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd80a5e7-6d30-4999-ad63-44a9a6413e82 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:06.505905 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e1c0f3-e225-41bc-8887-5d7b52fd259c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:06.506542 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1c0f3-e225-41bc-8887-5d7b52fd259c None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:06.506945 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1c0f3-e225-41bc-8887-5d7b52fd259c None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:06.507226 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1c0f3-e225-41bc-8887-5d7b52fd259c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:06.508887 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e1c0f3-e225-41bc-8887-5d7b52fd259c None admin] RBAC: Adding query filter params (role.id=653150c294214a0da3bdd8d00aacd3aa, user.id=640b55004f3f4278b286d4f1e6ef2132, scope.system=all) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:06.509215 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e1c0f3-e225-41bc-8887-5d7b52fd259c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:06.513628 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e1c0f3-e225-41bc-8887-5d7b52fd259c None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:06.527401 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 180/369] 10.222.0.42 () {56 vars in 1308 bytes} [Thu Sep 8 01:00:06 2022] GET /identity/v3/role_assignments?user.id=640b55004f3f4278b286d4f1e6ef2132&scope.system=all&role.id=653150c294214a0da3bdd8d00aacd3aa => generated 510 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:06.529611 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e1c0f3-e225-41bc-8887-5d7b52fd259c None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:06.566047 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b0beca-6cca-4b7e-83bd-6fc79fabaea4 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:06.602533 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:06.629567 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c2295f-2866-43e5-9024-be7c053c1ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:06.630228 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c2295f-2866-43e5-9024-be7c053c1ef2 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:06.630482 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c2295f-2866-43e5-9024-be7c053c1ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:06.630746 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c2295f-2866-43e5-9024-be7c053c1ef2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:06.658993 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c2295f-2866-43e5-9024-be7c053c1ef2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:00:06.670331 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-e8c2295f-2866-43e5-9024-be7c053c1ef2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 01:00:06.670331 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:06.670331 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:06.670331 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:06.670331 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:06.670331 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:06.670331 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:06.670331 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:06.670331 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:06.670331 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:06.670331 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:06.670331 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:06.670331 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:00:06.670331 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:00:06.670331 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:00:06.670331 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:00:06.671304 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:00:06.671304 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:00:06.671304 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:00:06.671304 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:00:06.671304 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:06.671304 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:06.671304 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:06.671304 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:06.671304 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:06.671304 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:06.671304 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:06.671304 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:06.671304 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:06.671304 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:06.671304 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:06.671304 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:06.672093 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:06.672093 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:06.672093 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:06.672093 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:06.672093 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:00:06.672093 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:00:06.672093 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:00:06.672093 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:00:06.672093 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:00:06.672093 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:00:06.672093 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 01:00:06.672093 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:06.672093 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 190/370] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 8 01:00:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:06.673368 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c2295f-2866-43e5-9024-be7c053c1ef2 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:06.768300 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e1c0f3-e225-41bc-8887-5d7b52fd259c None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:06.830821 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7935d416-2e92-4397-94d0-d936a9168491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:06.831933 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7935d416-2e92-4397-94d0-d936a9168491 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:06.832515 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7935d416-2e92-4397-94d0-d936a9168491 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:06.833166 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:06.834167 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7935d416-2e92-4397-94d0-d936a9168491 None admin] PATH_INFO: `/v3/groups/0115efc9f6184a23bcf4d23ff19c2115` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:06.836454 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7935d416-2e92-4397-94d0-d936a9168491 None admin] RBAC: Authorizing `identity:get_group(group_id=0115efc9f6184a23bcf4d23ff19c2115)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:06.838465 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7935d416-2e92-4397-94d0-d936a9168491 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:06.841207 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 181/371] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Sep 8 01:00:06 2022] GET /identity/v3/groups/0115efc9f6184a23bcf4d23ff19c2115 => generated 226 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:06.842393 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7935d416-2e92-4397-94d0-d936a9168491 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:06.847198 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c2295f-2866-43e5-9024-be7c053c1ef2 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:06.891766 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd5764f-99b1-4b5f-81f9-840f60ca7518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:06.892151 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd5764f-99b1-4b5f-81f9-840f60ca7518 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:06.892440 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd5764f-99b1-4b5f-81f9-840f60ca7518 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:06.892705 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd5764f-99b1-4b5f-81f9-840f60ca7518 None admin] PATH_INFO: `/v3/projects/0f19f058a73f49819b30c33d7fcfc52f` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:06.893849 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd5764f-99b1-4b5f-81f9-840f60ca7518 None admin] RBAC: Authorizing `identity:get_project(project_id=0f19f058a73f49819b30c33d7fcfc52f)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:06.895284 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd5764f-99b1-4b5f-81f9-840f60ca7518 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:06.899703 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 191/372] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 01:00:06 2022] GET /identity/v3/projects/0f19f058a73f49819b30c33d7fcfc52f => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:06.899703 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd5764f-99b1-4b5f-81f9-840f60ca7518 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:06.971612 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:06.975706 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7935d416-2e92-4397-94d0-d936a9168491 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:07.014375 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff4faf8-32cf-4f7d-93e0-f197fac6d993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:07.015408 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4faf8-32cf-4f7d-93e0-f197fac6d993 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:07.015912 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4faf8-32cf-4f7d-93e0-f197fac6d993 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:07.016364 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd5764f-99b1-4b5f-81f9-840f60ca7518 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:07.016662 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4faf8-32cf-4f7d-93e0-f197fac6d993 None admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:07.017537 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff4faf8-32cf-4f7d-93e0-f197fac6d993 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:07.018038 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff4faf8-32cf-4f7d-93e0-f197fac6d993 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:07.019723 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff4faf8-32cf-4f7d-93e0-f197fac6d993 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:07.030747 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 182/373] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 01:00:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:07.032012 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff4faf8-32cf-4f7d-93e0-f197fac6d993 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:07.054240 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77839931-aa1e-4aa5-b326-cd467c91f433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:07.054830 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77839931-aa1e-4aa5-b326-cd467c91f433 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:07.055105 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77839931-aa1e-4aa5-b326-cd467c91f433 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:07.055374 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77839931-aa1e-4aa5-b326-cd467c91f433 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:07.056095 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77839931-aa1e-4aa5-b326-cd467c91f433 None admin] RBAC: Adding query filter params (group.id=0115efc9f6184a23bcf4d23ff19c2115, role.id=92aa6d6824404d10b4a3be8b4a6bc317, scope.project.id=0f19f058a73f49819b30c33d7fcfc52f) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:07.056428 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77839931-aa1e-4aa5-b326-cd467c91f433 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:07.057984 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77839931-aa1e-4aa5-b326-cd467c91f433 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:07.065130 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 192/374] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Sep 8 01:00:06 2022] GET /identity/v3/role_assignments?group.id=0115efc9f6184a23bcf4d23ff19c2115&scope.project.id=0f19f058a73f49819b30c33d7fcfc52f&role.id=92aa6d6824404d10b4a3be8b4a6bc317 => generated 260 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:07.065529 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77839931-aa1e-4aa5-b326-cd467c91f433 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:07.091538 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:07.147898 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff4faf8-32cf-4f7d-93e0-f197fac6d993 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:07.185220 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77839931-aa1e-4aa5-b326-cd467c91f433 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:07.186162 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e942156-88a8-4bbe-a5a3-e6ccca7b8387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:07.187082 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e942156-88a8-4bbe-a5a3-e6ccca7b8387 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:07.187589 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e942156-88a8-4bbe-a5a3-e6ccca7b8387 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:07.188064 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e942156-88a8-4bbe-a5a3-e6ccca7b8387 None admin] PATH_INFO: `/v3/groups/0115efc9f6184a23bcf4d23ff19c2115` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:07.189443 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e942156-88a8-4bbe-a5a3-e6ccca7b8387 None admin] RBAC: Authorizing `identity:get_group(group_id=0115efc9f6184a23bcf4d23ff19c2115)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:07.191156 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e942156-88a8-4bbe-a5a3-e6ccca7b8387 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:07.193595 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 183/375] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Sep 8 01:00:07 2022] GET /identity/v3/groups/0115efc9f6184a23bcf4d23ff19c2115 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:07.213590 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e942156-88a8-4bbe-a5a3-e6ccca7b8387 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:07.230706 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c8c447-531b-4539-bbf8-745356556b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:07.231435 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c8c447-531b-4539-bbf8-745356556b76 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:07.232352 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c8c447-531b-4539-bbf8-745356556b76 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:07.232634 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c8c447-531b-4539-bbf8-745356556b76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:07.265634 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:07.316547 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c8c447-531b-4539-bbf8-745356556b76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:07.317959 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c8c447-531b-4539-bbf8-745356556b76 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:07.341693 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:07.355243 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e942156-88a8-4bbe-a5a3-e6ccca7b8387 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:07.396590 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a4abc9-aae7-4443-a373-e5de1a38c8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:07.397713 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4abc9-aae7-4443-a373-e5de1a38c8a0 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:07.398195 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4abc9-aae7-4443-a373-e5de1a38c8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:07.398702 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4abc9-aae7-4443-a373-e5de1a38c8a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:07.399645 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a4abc9-aae7-4443-a373-e5de1a38c8a0 None admin] RBAC: Adding query filter params (group.id=0115efc9f6184a23bcf4d23ff19c2115, role.id=069fe3e033664493b3def844e3735bc9, scope.project.id=0f19f058a73f49819b30c33d7fcfc52f) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:07.400193 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a4abc9-aae7-4443-a373-e5de1a38c8a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:07.401890 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a4abc9-aae7-4443-a373-e5de1a38c8a0 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:07.409263 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 184/376] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Sep 8 01:00:07 2022] GET /identity/v3/role_assignments?group.id=0115efc9f6184a23bcf4d23ff19c2115&scope.project.id=0f19f058a73f49819b30c33d7fcfc52f&role.id=069fe3e033664493b3def844e3735bc9 => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:00:07.455182 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 193/377] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 01:00:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 01:00:07.461348 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a4abc9-aae7-4443-a373-e5de1a38c8a0 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:07.555468 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5226d9a-9920-423e-af84-b2de823e929d None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:07.555816 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5226d9a-9920-423e-af84-b2de823e929d None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:07.556041 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5226d9a-9920-423e-af84-b2de823e929d None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:07.557695 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 194/378] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:00:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:00:07.563780 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e915379-13e9-48c7-a74c-a8f0db2cce19 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:07.564056 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e915379-13e9-48c7-a74c-a8f0db2cce19 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:07.564359 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e915379-13e9-48c7-a74c-a8f0db2cce19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:07.587315 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-6e915379-13e9-48c7-a74c-a8f0db2cce19 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:00:07.588602 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-6e915379-13e9-48c7-a74c-a8f0db2cce19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:00:07.600548 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:07.613707 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a4abc9-aae7-4443-a373-e5de1a38c8a0 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:07.681479 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d15493e-c8e5-4be8-b430-348fda0fe455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:07.683379 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d15493e-c8e5-4be8-b430-348fda0fe455 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:07.684046 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d15493e-c8e5-4be8-b430-348fda0fe455 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:07.686651 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d15493e-c8e5-4be8-b430-348fda0fe455 None admin] PATH_INFO: `/v3/projects/fc3cae79b39a4ab893f53897bb099478` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:07.688160 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d15493e-c8e5-4be8-b430-348fda0fe455 None admin] RBAC: Authorizing `identity:get_project(project_id=fc3cae79b39a4ab893f53897bb099478)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:07.689866 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d15493e-c8e5-4be8-b430-348fda0fe455 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:07.695827 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 185/379] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 01:00:07 2022] GET /identity/v3/projects/fc3cae79b39a4ab893f53897bb099478 => generated 344 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:07.701483 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d15493e-c8e5-4be8-b430-348fda0fe455 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:07.744282 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 195/380] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:00:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 01:00:07.831504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d15493e-c8e5-4be8-b430-348fda0fe455 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:07.866945 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0283a565-77f9-4883-9f0b-31e548d8215b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:07.867521 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283a565-77f9-4883-9f0b-31e548d8215b None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:07.867804 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283a565-77f9-4883-9f0b-31e548d8215b None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:07.868147 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283a565-77f9-4883-9f0b-31e548d8215b None admin] PATH_INFO: `/v3/groups/333dc8aa4d294de79d04753d2de626a2` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:07.874190 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0283a565-77f9-4883-9f0b-31e548d8215b None admin] RBAC: Authorizing `identity:get_group(group_id=333dc8aa4d294de79d04753d2de626a2)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:07.875728 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0283a565-77f9-4883-9f0b-31e548d8215b None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:07.877425 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 186/381] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Sep 8 01:00:07 2022] GET /identity/v3/groups/333dc8aa4d294de79d04753d2de626a2 => generated 229 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:07.883423 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e915379-13e9-48c7-a74c-a8f0db2cce19 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:07.945626 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:08.005980 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e915379-13e9-48c7-a74c-a8f0db2cce19 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:08.045485 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5a22fe-2d19-46e9-97a4-67b134aef209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:08.046432 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a22fe-2d19-46e9-97a4-67b134aef209 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:08.046905 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a22fe-2d19-46e9-97a4-67b134aef209 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:08.047372 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a22fe-2d19-46e9-97a4-67b134aef209 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:08.048345 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5a22fe-2d19-46e9-97a4-67b134aef209 None admin] RBAC: Adding query filter params (group.id=333dc8aa4d294de79d04753d2de626a2, role.id=34de369ec7c54715aa83a9052a26c3f2, scope.project.id=fc3cae79b39a4ab893f53897bb099478) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:08.048869 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5a22fe-2d19-46e9-97a4-67b134aef209 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:08.050510 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5a22fe-2d19-46e9-97a4-67b134aef209 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:08.058420 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0283a565-77f9-4883-9f0b-31e548d8215b None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:08.060973 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 196/382] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Sep 8 01:00:07 2022] GET /identity/v3/role_assignments?group.id=333dc8aa4d294de79d04753d2de626a2&scope.project.id=fc3cae79b39a4ab893f53897bb099478&role.id=34de369ec7c54715aa83a9052a26c3f2 => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:08.107384 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:08.207539 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0283a565-77f9-4883-9f0b-31e548d8215b None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:08.249571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e22ecf-6a7b-4d2f-873e-454e917c9f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:08.250231 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e22ecf-6a7b-4d2f-873e-454e917c9f37 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:08.250507 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e22ecf-6a7b-4d2f-873e-454e917c9f37 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:08.250777 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e22ecf-6a7b-4d2f-873e-454e917c9f37 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:08.266442 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e22ecf-6a7b-4d2f-873e-454e917c9f37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:00:08.272091 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-10e22ecf-6a7b-4d2f-873e-454e917c9f37 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 01:00:08.272091 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:08.272091 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:08.272091 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:08.272091 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:08.272091 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:08.272091 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:08.272091 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:08.272091 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:08.272091 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:08.272091 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:08.272091 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:08.272091 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:00:08.272091 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:00:08.272091 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:00:08.272091 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:00:08.273064 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:00:08.273064 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:00:08.273064 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:00:08.273064 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:00:08.273064 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:08.273064 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:08.273064 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:08.273064 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:08.273064 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:08.273064 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:08.273064 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:08.273064 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:08.273064 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:08.273064 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:08.273064 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:08.273064 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:08.273868 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:08.273868 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:08.273868 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:08.273868 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:08.273868 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:00:08.273868 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:00:08.273868 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:00:08.273868 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:00:08.273868 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:00:08.273868 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:00:08.273868 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 01:00:08.273868 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:08.273868 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 187/383] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 01:00:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:08.279271 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5a22fe-2d19-46e9-97a4-67b134aef209 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:08.415943 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5a22fe-2d19-46e9-97a4-67b134aef209 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:08.436158 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:08.458278 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49b1e1e-0061-42ca-bd24-bcee968dd480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:08.458809 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49b1e1e-0061-42ca-bd24-bcee968dd480 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:08.459088 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49b1e1e-0061-42ca-bd24-bcee968dd480 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:08.459361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49b1e1e-0061-42ca-bd24-bcee968dd480 None admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:08.460026 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49b1e1e-0061-42ca-bd24-bcee968dd480 None admin] RBAC: Adding query filter params (name=member) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:08.460355 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49b1e1e-0061-42ca-bd24-bcee968dd480 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:08.461728 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49b1e1e-0061-42ca-bd24-bcee968dd480 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:08.475921 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 197/384] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 8 01:00:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:08.482058 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e22ecf-6a7b-4d2f-873e-454e917c9f37 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:08.600593 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:08.619693 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e22ecf-6a7b-4d2f-873e-454e917c9f37 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:08.665318 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f769ecc8-b35d-47a4-aebd-be355ae5be62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:08.665956 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f769ecc8-b35d-47a4-aebd-be355ae5be62 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:08.666231 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f769ecc8-b35d-47a4-aebd-be355ae5be62 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:08.666671 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f769ecc8-b35d-47a4-aebd-be355ae5be62 None admin] PATH_INFO: `/v3/groups/0115efc9f6184a23bcf4d23ff19c2115` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:08.672852 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f769ecc8-b35d-47a4-aebd-be355ae5be62 None admin] RBAC: Authorizing `identity:get_group(group_id=0115efc9f6184a23bcf4d23ff19c2115)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:08.674324 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f769ecc8-b35d-47a4-aebd-be355ae5be62 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:08.676121 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 188/385] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Sep 8 01:00:08 2022] GET /identity/v3/groups/0115efc9f6184a23bcf4d23ff19c2115 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:08.681554 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49b1e1e-0061-42ca-bd24-bcee968dd480 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:08.821945 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49b1e1e-0061-42ca-bd24-bcee968dd480 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:08.866726 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecd1cfc-ccd1-4755-98e4-229f2c7ee8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:08.867391 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecd1cfc-ccd1-4755-98e4-229f2c7ee8ff None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:08.867673 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecd1cfc-ccd1-4755-98e4-229f2c7ee8ff None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:08.867950 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecd1cfc-ccd1-4755-98e4-229f2c7ee8ff None admin] PATH_INFO: `/v3/projects/41f15a3cf54f4397920b0e8f758539ea` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:08.869255 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecd1cfc-ccd1-4755-98e4-229f2c7ee8ff None admin] RBAC: Authorizing `identity:get_project(project_id=41f15a3cf54f4397920b0e8f758539ea)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:08.870818 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecd1cfc-ccd1-4755-98e4-229f2c7ee8ff None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:08.872761 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 198/386] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 01:00:08 2022] GET /identity/v3/projects/41f15a3cf54f4397920b0e8f758539ea => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:08.878787 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f769ecc8-b35d-47a4-aebd-be355ae5be62 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:09.016591 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b4d78-26c6-4c5f-8fa9-d2d9dbd8bd13 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:09.016908 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b4d78-26c6-4c5f-8fa9-d2d9dbd8bd13 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:09.017141 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b4d78-26c6-4c5f-8fa9-d2d9dbd8bd13 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:09.018419 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f769ecc8-b35d-47a4-aebd-be355ae5be62 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:09.018729 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 199/387] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:00:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 01:00:09.023619 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf38ab6-c946-4a9c-b7b4-62bc5baf1dd1 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:09.023902 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf38ab6-c946-4a9c-b7b4-62bc5baf1dd1 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:09.024192 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf38ab6-c946-4a9c-b7b4-62bc5baf1dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:09.045890 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-daf38ab6-c946-4a9c-b7b4-62bc5baf1dd1 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:00:09.047080 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-daf38ab6-c946-4a9c-b7b4-62bc5baf1dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:00:09.074439 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36813658-8c5d-484c-8d94-1edf536ed061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:09.075567 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36813658-8c5d-484c-8d94-1edf536ed061 None admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:09.075897 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36813658-8c5d-484c-8d94-1edf536ed061 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:09.076255 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36813658-8c5d-484c-8d94-1edf536ed061 None admin] PATH_INFO: `/v3/projects/41f15a3cf54f4397920b0e8f758539ea/groups/0115efc9f6184a23bcf4d23ff19c2115/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:09.080890 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36813658-8c5d-484c-8d94-1edf536ed061 None admin] RBAC: Authorizing `identity:create_grant(project_id=41f15a3cf54f4397920b0e8f758539ea, group_id=0115efc9f6184a23bcf4d23ff19c2115, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:09.082404 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36813658-8c5d-484c-8d94-1edf536ed061 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:09.102630 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 189/388] 10.222.0.42 () {58 vars in 1442 bytes} [Thu Sep 8 01:00:08 2022] PUT /identity/v3/projects/41f15a3cf54f4397920b0e8f758539ea/groups/0115efc9f6184a23bcf4d23ff19c2115/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:00:09.113627 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:09.147575 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 200/389] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:00:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 01:00:09.595178 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36813658-8c5d-484c-8d94-1edf536ed061 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:09.621533 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:09.746558 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36813658-8c5d-484c-8d94-1edf536ed061 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:09.795698 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946c0019-3f97-439b-93ee-d7f3f857eb00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:09.796357 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946c0019-3f97-439b-93ee-d7f3f857eb00 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:09.796636 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946c0019-3f97-439b-93ee-d7f3f857eb00 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:09.796907 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946c0019-3f97-439b-93ee-d7f3f857eb00 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:09.812740 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946c0019-3f97-439b-93ee-d7f3f857eb00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:00:09.818077 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-946c0019-3f97-439b-93ee-d7f3f857eb00 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 01:00:09.818077 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:09.818077 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:09.818077 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:09.818077 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:09.818077 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:09.818077 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:09.818077 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:09.818077 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:09.818077 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:09.818077 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:09.818077 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:09.818077 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:00:09.818077 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:00:09.818077 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:00:09.818077 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:00:09.818965 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:00:09.818965 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:00:09.818965 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:00:09.818965 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:00:09.818965 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:09.818965 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:09.818965 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:09.818965 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:09.818965 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:09.818965 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:09.818965 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:09.818965 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:09.818965 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:09.818965 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:09.818965 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:09.818965 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:09.819745 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:09.819745 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:09.819745 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:09.819745 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:09.819745 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:00:09.819745 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:00:09.819745 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:00:09.819745 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:00:09.819745 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:00:09.819745 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:00:09.819745 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 01:00:09.819745 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:09.819745 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 190/390] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 8 01:00:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:09.826004 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf38ab6-c946-4a9c-b7b4-62bc5baf1dd1 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:09.976263 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf38ab6-c946-4a9c-b7b4-62bc5baf1dd1 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:09.996586 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61245a9-a505-4120-87c9-be93af718449 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:09.998322 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61245a9-a505-4120-87c9-be93af718449 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:09.998798 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61245a9-a505-4120-87c9-be93af718449 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:10.000713 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 191/391] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:00:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:00:10.005737 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317470eb-04a4-4ff4-8f45-9eb812ceea00 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:10.006235 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317470eb-04a4-4ff4-8f45-9eb812ceea00 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:10.006704 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317470eb-04a4-4ff4-8f45-9eb812ceea00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:10.012264 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5446eb-95be-4fd6-8d37-f10e1f779e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:10.012914 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5446eb-95be-4fd6-8d37-f10e1f779e21 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:10.013195 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5446eb-95be-4fd6-8d37-f10e1f779e21 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:10.013564 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5446eb-95be-4fd6-8d37-f10e1f779e21 None admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:10.014143 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5446eb-95be-4fd6-8d37-f10e1f779e21 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:10.014449 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5446eb-95be-4fd6-8d37-f10e1f779e21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:10.015909 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5446eb-95be-4fd6-8d37-f10e1f779e21 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:10.024367 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 201/392] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 01:00:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:10.029935 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-317470eb-04a4-4ff4-8f45-9eb812ceea00 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:00:10.030819 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5446eb-95be-4fd6-8d37-f10e1f779e21 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:10.031594 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-317470eb-04a4-4ff4-8f45-9eb812ceea00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:00:10.084343 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:10.155499 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 192/393] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:00:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 01:00:10.215886 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5446eb-95be-4fd6-8d37-f10e1f779e21 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:10.266640 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66589c8f-a456-444e-8659-ef77874b1ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:10.267514 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66589c8f-a456-444e-8659-ef77874b1ef3 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:10.267928 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66589c8f-a456-444e-8659-ef77874b1ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:10.268354 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66589c8f-a456-444e-8659-ef77874b1ef3 None admin] PATH_INFO: `/v3/groups/0115efc9f6184a23bcf4d23ff19c2115` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:10.269641 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66589c8f-a456-444e-8659-ef77874b1ef3 None admin] RBAC: Authorizing `identity:get_group(group_id=0115efc9f6184a23bcf4d23ff19c2115)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:10.271208 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66589c8f-a456-444e-8659-ef77874b1ef3 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:10.273056 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 202/394] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Sep 8 01:00:10 2022] GET /identity/v3/groups/0115efc9f6184a23bcf4d23ff19c2115 => generated 226 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:00:10.279111 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317470eb-04a4-4ff4-8f45-9eb812ceea00 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:10.283501 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:10.407124 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317470eb-04a4-4ff4-8f45-9eb812ceea00 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:10.447497 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496db137-eb68-4c29-8d65-69d9bb055342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:10.448386 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496db137-eb68-4c29-8d65-69d9bb055342 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:10.448803 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496db137-eb68-4c29-8d65-69d9bb055342 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:10.449232 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496db137-eb68-4c29-8d65-69d9bb055342 None admin] PATH_INFO: `/v3/projects/41f15a3cf54f4397920b0e8f758539ea` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:10.451364 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496db137-eb68-4c29-8d65-69d9bb055342 None admin] RBAC: Authorizing `identity:get_project(project_id=41f15a3cf54f4397920b0e8f758539ea)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:10.452993 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496db137-eb68-4c29-8d65-69d9bb055342 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:10.455054 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 193/395] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 01:00:10 2022] GET /identity/v3/projects/41f15a3cf54f4397920b0e8f758539ea => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:10.460400 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66589c8f-a456-444e-8659-ef77874b1ef3 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:10.469788 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496db137-eb68-4c29-8d65-69d9bb055342 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:10.597111 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:10.640300 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66589c8f-a456-444e-8659-ef77874b1ef3 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:10.643354 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:10.657066 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496db137-eb68-4c29-8d65-69d9bb055342 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:10.691676 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc76dc1-0d27-4e85-b719-354c91710c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:10.692815 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc76dc1-0d27-4e85-b719-354c91710c96 None admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:10.693407 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc76dc1-0d27-4e85-b719-354c91710c96 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:10.693881 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc76dc1-0d27-4e85-b719-354c91710c96 None admin] PATH_INFO: `/v3/projects/41f15a3cf54f4397920b0e8f758539ea/groups/0115efc9f6184a23bcf4d23ff19c2115/roles/069fe3e033664493b3def844e3735bc9` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:10.696471 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc76dc1-0d27-4e85-b719-354c91710c96 None admin] RBAC: Authorizing `identity:create_grant(project_id=41f15a3cf54f4397920b0e8f758539ea, group_id=0115efc9f6184a23bcf4d23ff19c2115, role_id=069fe3e033664493b3def844e3735bc9)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:10.698277 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc76dc1-0d27-4e85-b719-354c91710c96 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:10.720292 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 203/396] 10.222.0.42 () {58 vars in 1442 bytes} [Thu Sep 8 01:00:10 2022] PUT /identity/v3/projects/41f15a3cf54f4397920b0e8f758539ea/groups/0115efc9f6184a23bcf4d23ff19c2115/roles/069fe3e033664493b3def844e3735bc9 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:00:10.727937 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f090ac-858a-4794-ab9b-6ac6e648af52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:10.728631 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f090ac-858a-4794-ab9b-6ac6e648af52 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:10.728909 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f090ac-858a-4794-ab9b-6ac6e648af52 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:10.729177 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f090ac-858a-4794-ab9b-6ac6e648af52 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:10.742510 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f090ac-858a-4794-ab9b-6ac6e648af52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:00:10.748508 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-10f090ac-858a-4794-ab9b-6ac6e648af52 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 01:00:10.748508 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:10.748508 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:10.748508 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:10.748508 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:10.748508 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:10.748508 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:10.748508 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:10.748508 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:10.748508 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:10.748508 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:10.748508 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:10.748508 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:00:10.748508 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:00:10.748508 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:00:10.748508 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:00:10.749511 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:00:10.749511 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:00:10.749511 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:00:10.749511 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:00:10.749511 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:10.749511 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:10.749511 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:10.749511 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:10.749511 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:10.749511 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:10.749511 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:10.749511 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:10.749511 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:10.749511 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:10.749511 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:10.749511 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:10.750966 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:10.750966 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:10.750966 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:10.750966 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:10.750966 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:00:10.750966 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:00:10.750966 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:00:10.750966 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:00:10.750966 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:00:10.750966 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:00:10.750966 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 01:00:10.750966 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:10.750966 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 194/397] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 01:00:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 01:00:10.755692 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc76dc1-0d27-4e85-b719-354c91710c96 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:10.818272 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ce3e2-dc98-4576-8021-9c8a48e5e17d None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:10.818628 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ce3e2-dc98-4576-8021-9c8a48e5e17d None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:10.818842 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ce3e2-dc98-4576-8021-9c8a48e5e17d None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:10.821327 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 195/398] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:00:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:00:10.829648 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef29aea9-bf02-4c24-b99e-d0982371f3d1 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:10.829993 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef29aea9-bf02-4c24-b99e-d0982371f3d1 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:10.830199 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef29aea9-bf02-4c24-b99e-d0982371f3d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:10.867180 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-ef29aea9-bf02-4c24-b99e-d0982371f3d1 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:00:10.868363 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-ef29aea9-bf02-4c24-b99e-d0982371f3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:00:10.899942 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc76dc1-0d27-4e85-b719-354c91710c96 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:10.941134 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc77c45-e0bd-40b2-b0ac-ca60a0377fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:10.942184 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc77c45-e0bd-40b2-b0ac-ca60a0377fc6 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:10.942745 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc77c45-e0bd-40b2-b0ac-ca60a0377fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:10.943314 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc77c45-e0bd-40b2-b0ac-ca60a0377fc6 None admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:10.944221 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc77c45-e0bd-40b2-b0ac-ca60a0377fc6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:10.944747 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc77c45-e0bd-40b2-b0ac-ca60a0377fc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:10.946391 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc77c45-e0bd-40b2-b0ac-ca60a0377fc6 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:10.947558 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 196/399] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:00:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 01:00:10.956537 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649070e3-7c02-43bd-aec4-01b602cb6286 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:10.956841 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649070e3-7c02-43bd-aec4-01b602cb6286 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:10.957113 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649070e3-7c02-43bd-aec4-01b602cb6286 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:10.957563 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 204/400] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 8 01:00:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:10.958547 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 197/401] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:00:10 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:00:10.968182 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649070e3-7c02-43bd-aec4-01b602cb6286 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:10.968324 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d37eea2-04d2-48fa-8dbb-80f9619e487b None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:10.968324 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d37eea2-04d2-48fa-8dbb-80f9619e487b None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:10.968324 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d37eea2-04d2-48fa-8dbb-80f9619e487b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:10.989645 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-3d37eea2-04d2-48fa-8dbb-80f9619e487b None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:00:10.991020 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-3d37eea2-04d2-48fa-8dbb-80f9619e487b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:00:11.016394 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:11.039053 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:11.073473 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 205/402] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:00:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 01:00:11.113776 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649070e3-7c02-43bd-aec4-01b602cb6286 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:11.159139 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec083c-9217-408c-ae33-7b04453d1ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:11.159688 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec083c-9217-408c-ae33-7b04453d1ea5 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:11.159962 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec083c-9217-408c-ae33-7b04453d1ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:11.160261 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec083c-9217-408c-ae33-7b04453d1ea5 None admin] PATH_INFO: `/v3/groups/0115efc9f6184a23bcf4d23ff19c2115` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:11.161368 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ec083c-9217-408c-ae33-7b04453d1ea5 None admin] RBAC: Authorizing `identity:get_group(group_id=0115efc9f6184a23bcf4d23ff19c2115)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:11.162814 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ec083c-9217-408c-ae33-7b04453d1ea5 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:11.164392 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 198/403] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Sep 8 01:00:10 2022] GET /identity/v3/groups/0115efc9f6184a23bcf4d23ff19c2115 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:11.170656 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d37eea2-04d2-48fa-8dbb-80f9619e487b None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:11.289887 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec083c-9217-408c-ae33-7b04453d1ea5 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:11.340281 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d37eea2-04d2-48fa-8dbb-80f9619e487b None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:11.407637 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec083c-9217-408c-ae33-7b04453d1ea5 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:11.433496 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6728079-2c1e-4ebe-929b-f1c17867af98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:11.434069 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6728079-2c1e-4ebe-929b-f1c17867af98 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:11.434894 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6728079-2c1e-4ebe-929b-f1c17867af98 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:11.435169 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6728079-2c1e-4ebe-929b-f1c17867af98 None admin] PATH_INFO: `/v3/projects/0f19f058a73f49819b30c33d7fcfc52f` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:11.436324 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6728079-2c1e-4ebe-929b-f1c17867af98 None admin] RBAC: Authorizing `identity:get_project(project_id=0f19f058a73f49819b30c33d7fcfc52f)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:11.437764 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6728079-2c1e-4ebe-929b-f1c17867af98 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:11.439706 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 206/404] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 01:00:11 2022] GET /identity/v3/projects/0f19f058a73f49819b30c33d7fcfc52f => generated 302 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:11.445698 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:11.448262 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6728079-2c1e-4ebe-929b-f1c17867af98 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:11.451522 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79cb036-aa63-4b56-ab81-d3325edeea56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:11.452432 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79cb036-aa63-4b56-ab81-d3325edeea56 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:11.452923 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79cb036-aa63-4b56-ab81-d3325edeea56 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:11.453397 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79cb036-aa63-4b56-ab81-d3325edeea56 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:11.464553 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79cb036-aa63-4b56-ab81-d3325edeea56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:00:11.471337 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-c79cb036-aa63-4b56-ab81-d3325edeea56 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 01:00:11.471337 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:11.471337 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:11.471337 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:11.471337 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:11.471337 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:11.471337 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:11.471337 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:11.471337 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:11.471337 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:11.471337 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:11.471337 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:11.471337 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:00:11.471337 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:00:11.471337 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:00:11.471337 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:00:11.472304 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:00:11.472304 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:00:11.472304 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:00:11.472304 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:00:11.472304 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:11.472304 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:11.472304 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:11.472304 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:11.472304 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:11.472304 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:11.472304 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:11.472304 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:11.472304 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:11.472304 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:11.472304 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:11.472304 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:11.473082 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:11.473082 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:11.473082 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:11.473082 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:11.473082 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:00:11.473082 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:00:11.473082 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:00:11.473082 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:00:11.473082 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:00:11.473082 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:00:11.473082 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 01:00:11.473082 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:11.474549 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 199/405] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 8 01:00:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:11.485102 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79cb036-aa63-4b56-ab81-d3325edeea56 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:11.495927 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:11.586794 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6728079-2c1e-4ebe-929b-f1c17867af98 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:11.601700 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79cb036-aa63-4b56-ab81-d3325edeea56 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:11.630157 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b0a090-73a5-45a4-9487-bb1e852e160b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:11.631360 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b0a090-73a5-45a4-9487-bb1e852e160b None admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:11.632161 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b0a090-73a5-45a4-9487-bb1e852e160b None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:11.632875 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b0a090-73a5-45a4-9487-bb1e852e160b None admin] PATH_INFO: `/v3/projects/0f19f058a73f49819b30c33d7fcfc52f/groups/0115efc9f6184a23bcf4d23ff19c2115/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:11.635395 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b0a090-73a5-45a4-9487-bb1e852e160b None admin] RBAC: Authorizing `identity:create_grant(project_id=0f19f058a73f49819b30c33d7fcfc52f, group_id=0115efc9f6184a23bcf4d23ff19c2115, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:11.637723 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b0a090-73a5-45a4-9487-bb1e852e160b None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:11.640554 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edfa0ed-27d4-4898-b8ae-e0f1949c89d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:11.644159 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edfa0ed-27d4-4898-b8ae-e0f1949c89d9 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:11.644159 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edfa0ed-27d4-4898-b8ae-e0f1949c89d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:11.644159 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edfa0ed-27d4-4898-b8ae-e0f1949c89d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:11.644159 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edfa0ed-27d4-4898-b8ae-e0f1949c89d9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:11.644159 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edfa0ed-27d4-4898-b8ae-e0f1949c89d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:11.646661 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edfa0ed-27d4-4898-b8ae-e0f1949c89d9 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:11.655802 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 207/406] 10.222.0.42 () {58 vars in 1442 bytes} [Thu Sep 8 01:00:11 2022] PUT /identity/v3/projects/0f19f058a73f49819b30c33d7fcfc52f/groups/0115efc9f6184a23bcf4d23ff19c2115/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:00:11.657835 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 200/407] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 01:00:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:11.665343 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b0a090-73a5-45a4-9487-bb1e852e160b None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:11.670732 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edfa0ed-27d4-4898-b8ae-e0f1949c89d9 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:11.790400 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:11.864176 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:11.881507 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b0a090-73a5-45a4-9487-bb1e852e160b None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:11.890401 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edfa0ed-27d4-4898-b8ae-e0f1949c89d9 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:11.929187 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d5c44-0957-4149-b4a4-38df963df4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:11.929809 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d5c44-0957-4149-b4a4-38df963df4b2 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:11.930092 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d5c44-0957-4149-b4a4-38df963df4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:11.930366 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d5c44-0957-4149-b4a4-38df963df4b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:11.930466 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc597de2-29d5-4976-b83d-0c3cfbdac80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:11.931364 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc597de2-29d5-4976-b83d-0c3cfbdac80b None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:11.931705 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc597de2-29d5-4976-b83d-0c3cfbdac80b None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:11.932052 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc597de2-29d5-4976-b83d-0c3cfbdac80b None admin] PATH_INFO: `/v3/groups/0115efc9f6184a23bcf4d23ff19c2115` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:11.933295 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc597de2-29d5-4976-b83d-0c3cfbdac80b None admin] RBAC: Authorizing `identity:get_group(group_id=0115efc9f6184a23bcf4d23ff19c2115)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:11.935887 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc597de2-29d5-4976-b83d-0c3cfbdac80b None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:11.938516 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 201/408] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Sep 8 01:00:11 2022] GET /identity/v3/groups/0115efc9f6184a23bcf4d23ff19c2115 => generated 226 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:00:11.939733 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c2b55b-18c8-44d6-b181-f99d1ddec013 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:11.940080 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c2b55b-18c8-44d6-b181-f99d1ddec013 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:11.940503 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c2b55b-18c8-44d6-b181-f99d1ddec013 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:11.941895 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 202/409] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:00:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:00:11.942389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3d5c44-0957-4149-b4a4-38df963df4b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:00:11.946533 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c2b55b-18c8-44d6-b181-f99d1ddec013 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:11.947530 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-ff3d5c44-0957-4149-b4a4-38df963df4b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 01:00:11.947530 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:11.947530 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:11.947530 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:11.947530 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:11.947530 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:11.947530 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:11.947530 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:11.947530 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:11.947530 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:11.947530 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:11.947530 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:11.947530 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:00:11.947530 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:00:11.947530 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:00:11.947530 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:00:11.948428 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:00:11.948428 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:00:11.948428 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:00:11.948428 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:00:11.948428 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:11.948428 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:11.948428 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:11.948428 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:11.948428 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:11.948428 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:11.948428 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:11.948428 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:11.948428 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:11.948428 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:11.948428 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:11.948428 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:11.949231 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:11.949231 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:11.949231 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:11.949231 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:11.949231 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:00:11.949231 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:00:11.949231 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:00:11.949231 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:00:11.949231 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:00:11.949231 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:00:11.949231 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 01:00:11.949231 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:11.949231 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 208/410] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 8 01:00:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:11.950890 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1f9fe-388b-4c8c-9347-a2bcf2431cde None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:11.953733 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1f9fe-388b-4c8c-9347-a2bcf2431cde None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:11.953733 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1f9fe-388b-4c8c-9347-a2bcf2431cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:11.976293 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-54a1f9fe-388b-4c8c-9347-a2bcf2431cde None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:00:11.977865 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-54a1f9fe-388b-4c8c-9347-a2bcf2431cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:00:12.061202 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 209/411] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:00:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 01:00:12.062175 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a1f9fe-388b-4c8c-9347-a2bcf2431cde None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:12.070626 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c2b55b-18c8-44d6-b181-f99d1ddec013 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:12.105524 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:12.132939 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee20c33-2879-4689-b88e-218d5e5da4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:12.133605 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee20c33-2879-4689-b88e-218d5e5da4c2 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:12.133872 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee20c33-2879-4689-b88e-218d5e5da4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:12.134137 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee20c33-2879-4689-b88e-218d5e5da4c2 None admin] PATH_INFO: `/v3/projects/0f19f058a73f49819b30c33d7fcfc52f` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:12.137986 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee20c33-2879-4689-b88e-218d5e5da4c2 None admin] RBAC: Authorizing `identity:get_project(project_id=0f19f058a73f49819b30c33d7fcfc52f)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:12.139625 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee20c33-2879-4689-b88e-218d5e5da4c2 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:12.141914 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 203/412] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 01:00:11 2022] GET /identity/v3/projects/0f19f058a73f49819b30c33d7fcfc52f => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:12.147456 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee20c33-2879-4689-b88e-218d5e5da4c2 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:12.149999 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:12.178621 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a1f9fe-388b-4c8c-9347-a2bcf2431cde None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:12.225614 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6a5a50-b3aa-40d9-9fef-c79ec175abf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:12.226481 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6a5a50-b3aa-40d9-9fef-c79ec175abf5 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:12.227181 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6a5a50-b3aa-40d9-9fef-c79ec175abf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:12.228488 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6a5a50-b3aa-40d9-9fef-c79ec175abf5 None admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:12.230240 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6a5a50-b3aa-40d9-9fef-c79ec175abf5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:12.230773 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6a5a50-b3aa-40d9-9fef-c79ec175abf5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:12.232648 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6a5a50-b3aa-40d9-9fef-c79ec175abf5 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:12.244459 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 210/413] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 01:00:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:12.250837 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6a5a50-b3aa-40d9-9fef-c79ec175abf5 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:12.361720 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee20c33-2879-4689-b88e-218d5e5da4c2 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:12.436419 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be9e86e-6ff4-4046-984b-3e44daf19947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:12.437332 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9e86e-6ff4-4046-984b-3e44daf19947 None admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:12.437909 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9e86e-6ff4-4046-984b-3e44daf19947 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:12.438376 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9e86e-6ff4-4046-984b-3e44daf19947 None admin] PATH_INFO: `/v3/projects/0f19f058a73f49819b30c33d7fcfc52f/groups/0115efc9f6184a23bcf4d23ff19c2115/roles/069fe3e033664493b3def844e3735bc9` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:12.440921 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be9e86e-6ff4-4046-984b-3e44daf19947 None admin] RBAC: Authorizing `identity:create_grant(project_id=0f19f058a73f49819b30c33d7fcfc52f, group_id=0115efc9f6184a23bcf4d23ff19c2115, role_id=069fe3e033664493b3def844e3735bc9)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:12.442571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be9e86e-6ff4-4046-984b-3e44daf19947 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:12.443927 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6a5a50-b3aa-40d9-9fef-c79ec175abf5 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:12.459004 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 204/414] 10.222.0.42 () {58 vars in 1442 bytes} [Thu Sep 8 01:00:12 2022] PUT /identity/v3/projects/0f19f058a73f49819b30c33d7fcfc52f/groups/0115efc9f6184a23bcf4d23ff19c2115/roles/069fe3e033664493b3def844e3735bc9 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:00:12.460206 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be9e86e-6ff4-4046-984b-3e44daf19947 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:12.506806 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f388e58-206a-4fed-8308-bd9536a120b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:12.507457 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f388e58-206a-4fed-8308-bd9536a120b5 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:12.507739 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f388e58-206a-4fed-8308-bd9536a120b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:12.508013 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f388e58-206a-4fed-8308-bd9536a120b5 None admin] PATH_INFO: `/v3/groups/333dc8aa4d294de79d04753d2de626a2` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:12.513130 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f388e58-206a-4fed-8308-bd9536a120b5 None admin] RBAC: Authorizing `identity:get_group(group_id=333dc8aa4d294de79d04753d2de626a2)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:12.514941 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f388e58-206a-4fed-8308-bd9536a120b5 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:12.517843 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 211/415] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Sep 8 01:00:12 2022] GET /identity/v3/groups/333dc8aa4d294de79d04753d2de626a2 => generated 229 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:00:12.518939 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:12.524599 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f388e58-206a-4fed-8308-bd9536a120b5 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:12.529738 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:12.613646 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be9e86e-6ff4-4046-984b-3e44daf19947 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:12.660979 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f388e58-206a-4fed-8308-bd9536a120b5 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:12.661177 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a3663-d80d-49dc-a120-fd9e76d3d0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:12.661874 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a3663-d80d-49dc-a120-fd9e76d3d0fd None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:12.662158 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a3663-d80d-49dc-a120-fd9e76d3d0fd None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:12.662430 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a3663-d80d-49dc-a120-fd9e76d3d0fd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:12.688987 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994a3663-d80d-49dc-a120-fd9e76d3d0fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:00:12.705458 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-994a3663-d80d-49dc-a120-fd9e76d3d0fd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 01:00:12.705458 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:12.705458 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:12.705458 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:12.705458 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:12.705458 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:12.705458 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:12.705458 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:12.705458 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:12.705458 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:12.705458 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:12.705458 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:12.705458 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:00:12.705458 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:00:12.705458 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:00:12.705458 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:00:12.706532 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:00:12.706532 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:00:12.706532 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:00:12.706532 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:00:12.706532 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:12.706532 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:12.706532 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:12.706532 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:12.706532 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:12.706532 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:12.706532 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:12.706532 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:12.706532 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:12.706532 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:12.706532 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:12.706532 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:12.707309 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:12.707309 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:12.707309 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:12.707309 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:12.707309 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:00:12.707309 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:00:12.707309 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:00:12.707309 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:00:12.707309 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:00:12.707309 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:00:12.707309 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 01:00:12.707309 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:12.710136 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 205/416] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 01:00:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:12.716621 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a3663-d80d-49dc-a120-fd9e76d3d0fd None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:12.721949 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda35f99-e7ec-4197-9882-0d2d02ef92db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:12.722636 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda35f99-e7ec-4197-9882-0d2d02ef92db None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:12.722928 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda35f99-e7ec-4197-9882-0d2d02ef92db None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:12.723208 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda35f99-e7ec-4197-9882-0d2d02ef92db None admin] PATH_INFO: `/v3/projects/fc3cae79b39a4ab893f53897bb099478` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:12.729148 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda35f99-e7ec-4197-9882-0d2d02ef92db None admin] RBAC: Authorizing `identity:get_project(project_id=fc3cae79b39a4ab893f53897bb099478)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:12.730690 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda35f99-e7ec-4197-9882-0d2d02ef92db None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:12.734800 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 212/417] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 01:00:12 2022] GET /identity/v3/projects/fc3cae79b39a4ab893f53897bb099478 => generated 344 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:00:12.741693 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda35f99-e7ec-4197-9882-0d2d02ef92db None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:12.832708 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a3663-d80d-49dc-a120-fd9e76d3d0fd None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:12.834230 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:12.850562 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:12.886866 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716988ad-c49a-47c0-a33d-1305884e5046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:12.887437 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716988ad-c49a-47c0-a33d-1305884e5046 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:12.887728 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716988ad-c49a-47c0-a33d-1305884e5046 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:12.887998 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716988ad-c49a-47c0-a33d-1305884e5046 None admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:12.888721 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716988ad-c49a-47c0-a33d-1305884e5046 None admin] RBAC: Adding query filter params (name=member) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:12.889000 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716988ad-c49a-47c0-a33d-1305884e5046 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:12.890406 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716988ad-c49a-47c0-a33d-1305884e5046 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:12.899027 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda35f99-e7ec-4197-9882-0d2d02ef92db None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:12.912564 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 206/418] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 8 01:00:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:00:12.920059 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716988ad-c49a-47c0-a33d-1305884e5046 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:12.973913 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61815636-209d-413d-a184-d2e29eaf0638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:12.975186 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61815636-209d-413d-a184-d2e29eaf0638 None admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:12.975623 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61815636-209d-413d-a184-d2e29eaf0638 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:12.975921 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61815636-209d-413d-a184-d2e29eaf0638 None admin] PATH_INFO: `/v3/projects/fc3cae79b39a4ab893f53897bb099478/groups/333dc8aa4d294de79d04753d2de626a2/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:12.978466 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61815636-209d-413d-a184-d2e29eaf0638 None admin] RBAC: Authorizing `identity:create_grant(project_id=fc3cae79b39a4ab893f53897bb099478, group_id=333dc8aa4d294de79d04753d2de626a2, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:12.979972 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61815636-209d-413d-a184-d2e29eaf0638 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:13.004674 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 213/419] 10.222.0.42 () {58 vars in 1442 bytes} [Thu Sep 8 01:00:12 2022] PUT /identity/v3/projects/fc3cae79b39a4ab893f53897bb099478/groups/333dc8aa4d294de79d04753d2de626a2/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:00:13.094866 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716988ad-c49a-47c0-a33d-1305884e5046 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:13.150639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78e1dea-3c74-40e5-808f-0a1493024e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:13.151554 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78e1dea-3c74-40e5-808f-0a1493024e07 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:13.152014 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78e1dea-3c74-40e5-808f-0a1493024e07 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:13.152449 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78e1dea-3c74-40e5-808f-0a1493024e07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:13.222154 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:13.268469 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78e1dea-3c74-40e5-808f-0a1493024e07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:13.269985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78e1dea-3c74-40e5-808f-0a1493024e07 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:13.338886 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ed4d0-b81b-4290-8f75-0f27d1f97f21 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:13.339227 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ed4d0-b81b-4290-8f75-0f27d1f97f21 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:13.339435 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ed4d0-b81b-4290-8f75-0f27d1f97f21 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:13.341359 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 214/420] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:00:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:00:13.352554 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcadb5c6-a898-4135-9c02-fdad1f47b1ae None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:13.352837 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcadb5c6-a898-4135-9c02-fdad1f47b1ae None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:13.353101 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcadb5c6-a898-4135-9c02-fdad1f47b1ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:13.374203 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-bcadb5c6-a898-4135-9c02-fdad1f47b1ae None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:00:13.375405 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-bcadb5c6-a898-4135-9c02-fdad1f47b1ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:00:13.409656 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 207/421] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 01:00:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 01:00:13.414732 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78e1dea-3c74-40e5-808f-0a1493024e07 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:13.473545 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 215/422] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:00:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 01:00:13.505847 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:13.542884 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78e1dea-3c74-40e5-808f-0a1493024e07 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:13.584602 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce55ff33-9af7-407e-909c-817a2f8100aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:13.585353 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce55ff33-9af7-407e-909c-817a2f8100aa None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:13.585632 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce55ff33-9af7-407e-909c-817a2f8100aa None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:13.585904 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce55ff33-9af7-407e-909c-817a2f8100aa None admin] PATH_INFO: `/v3/projects/41f15a3cf54f4397920b0e8f758539ea` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:13.587097 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce55ff33-9af7-407e-909c-817a2f8100aa None admin] RBAC: Authorizing `identity:get_project(project_id=41f15a3cf54f4397920b0e8f758539ea)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:13.588677 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce55ff33-9af7-407e-909c-817a2f8100aa None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:13.590857 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 208/423] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 01:00:13 2022] GET /identity/v3/projects/41f15a3cf54f4397920b0e8f758539ea => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:13.596693 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcadb5c6-a898-4135-9c02-fdad1f47b1ae None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:13.729311 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:13.793497 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcadb5c6-a898-4135-9c02-fdad1f47b1ae None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:13.849096 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7565f94-47b1-4c9f-91a9-24e5fd5d91cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:13.849730 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7565f94-47b1-4c9f-91a9-24e5fd5d91cf None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:13.850003 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7565f94-47b1-4c9f-91a9-24e5fd5d91cf None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:13.850269 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7565f94-47b1-4c9f-91a9-24e5fd5d91cf None admin] PATH_INFO: `/v3/groups/0115efc9f6184a23bcf4d23ff19c2115` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:13.851365 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7565f94-47b1-4c9f-91a9-24e5fd5d91cf None admin] RBAC: Authorizing `identity:get_group(group_id=0115efc9f6184a23bcf4d23ff19c2115)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:13.852841 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7565f94-47b1-4c9f-91a9-24e5fd5d91cf None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:13.854560 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 216/424] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Sep 8 01:00:13 2022] GET /identity/v3/groups/0115efc9f6184a23bcf4d23ff19c2115 => generated 226 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:13.871154 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce55ff33-9af7-407e-909c-817a2f8100aa None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:13.893005 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7565f94-47b1-4c9f-91a9-24e5fd5d91cf None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:14.002456 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce55ff33-9af7-407e-909c-817a2f8100aa None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:14.042432 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424de1a0-5325-4cab-b6fd-13ab3d34988e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:14.043039 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424de1a0-5325-4cab-b6fd-13ab3d34988e None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:14.043317 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424de1a0-5325-4cab-b6fd-13ab3d34988e None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:14.044176 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424de1a0-5325-4cab-b6fd-13ab3d34988e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:14.045064 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424de1a0-5325-4cab-b6fd-13ab3d34988e None admin] RBAC: Adding query filter params (group.id=0115efc9f6184a23bcf4d23ff19c2115, role.id=92aa6d6824404d10b4a3be8b4a6bc317, scope.project.id=41f15a3cf54f4397920b0e8f758539ea) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:14.046139 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424de1a0-5325-4cab-b6fd-13ab3d34988e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:14.047626 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424de1a0-5325-4cab-b6fd-13ab3d34988e None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:14.054737 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 209/425] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Sep 8 01:00:13 2022] GET /identity/v3/role_assignments?group.id=0115efc9f6184a23bcf4d23ff19c2115&scope.project.id=41f15a3cf54f4397920b0e8f758539ea&role.id=92aa6d6824404d10b4a3be8b4a6bc317 => generated 611 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:14.093644 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7565f94-47b1-4c9f-91a9-24e5fd5d91cf None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:14.127127 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:14.140324 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b77600f-c4e1-4f2f-9228-f06a8cee1c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:14.140979 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b77600f-c4e1-4f2f-9228-f06a8cee1c72 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:14.141315 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b77600f-c4e1-4f2f-9228-f06a8cee1c72 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:14.141571 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b77600f-c4e1-4f2f-9228-f06a8cee1c72 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:14.162193 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b77600f-c4e1-4f2f-9228-f06a8cee1c72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:00:14.167155 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-6b77600f-c4e1-4f2f-9228-f06a8cee1c72 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 01:00:14.167155 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:14.167155 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:14.167155 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:14.167155 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:14.167155 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:14.167155 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:14.167155 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:14.167155 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:14.167155 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:14.167155 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:14.167155 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:14.167155 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:00:14.167155 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:00:14.167155 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:00:14.167155 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:00:14.168049 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:00:14.168049 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:00:14.168049 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:00:14.168049 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:00:14.168049 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:14.168049 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:14.168049 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:14.168049 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:14.168049 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:14.168049 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:14.168049 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:14.168049 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:14.168049 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:14.168049 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:14.168049 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:14.168049 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:14.169432 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:14.169432 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:14.169432 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:14.169432 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:14.169432 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:00:14.169432 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:00:14.169432 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:00:14.169432 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:00:14.169432 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:00:14.169432 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:00:14.169432 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 01:00:14.169432 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:14.169432 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 217/426] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 8 01:00:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 01:00:14.175306 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424de1a0-5325-4cab-b6fd-13ab3d34988e None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:14.183133 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:14.241545 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbbe7e1-91fc-4fb0-b99f-ce56110a7212 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:14.241831 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbbe7e1-91fc-4fb0-b99f-ce56110a7212 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:14.242678 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbbe7e1-91fc-4fb0-b99f-ce56110a7212 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:14.246144 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 218/427] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:00:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:00:14.255098 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea15ac1-09ff-4fc9-b043-955cbb6ec042 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:14.255283 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea15ac1-09ff-4fc9-b043-955cbb6ec042 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:14.255560 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea15ac1-09ff-4fc9-b043-955cbb6ec042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:14.291351 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-6ea15ac1-09ff-4fc9-b043-955cbb6ec042 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:00:14.295030 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-6ea15ac1-09ff-4fc9-b043-955cbb6ec042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:00:14.350143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424de1a0-5325-4cab-b6fd-13ab3d34988e None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:14.377415 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 219/428] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:00:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 01:00:14.389592 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f50d029-bfb1-46d7-81f9-1308ba0a0c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:14.390371 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f50d029-bfb1-46d7-81f9-1308ba0a0c3c None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:14.390745 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f50d029-bfb1-46d7-81f9-1308ba0a0c3c None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:14.391135 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f50d029-bfb1-46d7-81f9-1308ba0a0c3c None admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:14.391929 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f50d029-bfb1-46d7-81f9-1308ba0a0c3c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:14.392380 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f50d029-bfb1-46d7-81f9-1308ba0a0c3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:14.393978 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f50d029-bfb1-46d7-81f9-1308ba0a0c3c None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:14.404781 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 210/429] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 01:00:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:14.411500 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea15ac1-09ff-4fc9-b043-955cbb6ec042 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:14.521944 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea15ac1-09ff-4fc9-b043-955cbb6ec042 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:14.543630 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:14.563551 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc6a43c-9f41-4b12-bf48-4eefaf298444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:14.564190 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6a43c-9f41-4b12-bf48-4eefaf298444 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:14.564456 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6a43c-9f41-4b12-bf48-4eefaf298444 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:14.564723 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6a43c-9f41-4b12-bf48-4eefaf298444 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:14.636431 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc6a43c-9f41-4b12-bf48-4eefaf298444 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:14.636942 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc6a43c-9f41-4b12-bf48-4eefaf298444 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:14.695596 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f50d029-bfb1-46d7-81f9-1308ba0a0c3c None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:14.754559 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 220/430] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 01:00:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 01:00:14.760010 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc6a43c-9f41-4b12-bf48-4eefaf298444 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:14.786748 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:14.800255 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:14.851033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f50d029-bfb1-46d7-81f9-1308ba0a0c3c None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:14.904849 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9becf9fa-48c5-4d4e-adf2-33a5d0be09f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:14.905757 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9becf9fa-48c5-4d4e-adf2-33a5d0be09f1 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:14.906371 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9becf9fa-48c5-4d4e-adf2-33a5d0be09f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:14.906652 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9becf9fa-48c5-4d4e-adf2-33a5d0be09f1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:14.913394 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc6a43c-9f41-4b12-bf48-4eefaf298444 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:14.920299 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9becf9fa-48c5-4d4e-adf2-33a5d0be09f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:00:14.926860 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-9becf9fa-48c5-4d4e-adf2-33a5d0be09f1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 01:00:14.926860 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:14.926860 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:14.926860 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:14.926860 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:14.926860 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:14.926860 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:14.926860 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:14.926860 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:14.926860 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:14.926860 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:14.926860 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:14.926860 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:00:14.926860 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:00:14.926860 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:00:14.926860 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:00:14.927744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:00:14.927744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:00:14.927744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:00:14.927744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:00:14.927744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:14.927744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:14.927744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:14.927744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:14.927744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:14.927744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:14.927744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:14.927744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:14.927744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:14.927744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:14.927744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:14.927744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:14.928571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:14.928571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:14.928571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:14.928571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:14.928571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:00:14.928571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:00:14.928571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:00:14.928571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:00:14.928571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:00:14.928571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:00:14.928571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 01:00:14.928571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:14.929385 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 211/431] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 01:00:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:14.935511 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9becf9fa-48c5-4d4e-adf2-33a5d0be09f1 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:14.966595 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bc4589-5b95-4d40-b2fa-8fd0a20534ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:14.967318 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc4589-5b95-4d40-b2fa-8fd0a20534ee None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:14.967659 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc4589-5b95-4d40-b2fa-8fd0a20534ee None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:14.968493 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc4589-5b95-4d40-b2fa-8fd0a20534ee None admin] PATH_INFO: `/v3/projects/41f15a3cf54f4397920b0e8f758539ea` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:14.969922 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bc4589-5b95-4d40-b2fa-8fd0a20534ee None admin] RBAC: Authorizing `identity:get_project(project_id=41f15a3cf54f4397920b0e8f758539ea)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:14.971559 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bc4589-5b95-4d40-b2fa-8fd0a20534ee None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:14.973312 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 221/432] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 01:00:14 2022] GET /identity/v3/projects/41f15a3cf54f4397920b0e8f758539ea => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:00:14.978582 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bc4589-5b95-4d40-b2fa-8fd0a20534ee None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:15.067225 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9becf9fa-48c5-4d4e-adf2-33a5d0be09f1 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:15.078442 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:15.092860 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bc4589-5b95-4d40-b2fa-8fd0a20534ee None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:15.096527 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:15.127755 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba740a63-3436-4273-9006-89179b3839e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:15.128472 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba740a63-3436-4273-9006-89179b3839e8 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:15.128749 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba740a63-3436-4273-9006-89179b3839e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:15.129019 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba740a63-3436-4273-9006-89179b3839e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:15.129711 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba740a63-3436-4273-9006-89179b3839e8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:15.130081 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba740a63-3436-4273-9006-89179b3839e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:15.132538 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba740a63-3436-4273-9006-89179b3839e8 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:15.149572 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 212/433] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 8 01:00:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:15.152222 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b01788-28cb-449b-b06a-b8c5d13d5f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:15.152704 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b01788-28cb-449b-b06a-b8c5d13d5f37 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:15.153478 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b01788-28cb-449b-b06a-b8c5d13d5f37 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:15.153875 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b01788-28cb-449b-b06a-b8c5d13d5f37 None admin] PATH_INFO: `/v3/groups/0115efc9f6184a23bcf4d23ff19c2115` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:15.155496 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b01788-28cb-449b-b06a-b8c5d13d5f37 None admin] RBAC: Authorizing `identity:get_group(group_id=0115efc9f6184a23bcf4d23ff19c2115)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:15.157244 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b01788-28cb-449b-b06a-b8c5d13d5f37 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:15.158159 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba740a63-3436-4273-9006-89179b3839e8 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:15.164457 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 222/434] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Sep 8 01:00:14 2022] GET /identity/v3/groups/0115efc9f6184a23bcf4d23ff19c2115 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:00:15.166461 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b01788-28cb-449b-b06a-b8c5d13d5f37 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:15.272835 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba740a63-3436-4273-9006-89179b3839e8 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:15.290733 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b01788-28cb-449b-b06a-b8c5d13d5f37 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:15.309048 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc2a26-24ef-430e-879d-e87eef55ccc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:15.310255 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc2a26-24ef-430e-879d-e87eef55ccc6 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:15.310534 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc2a26-24ef-430e-879d-e87eef55ccc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:15.310797 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc2a26-24ef-430e-879d-e87eef55ccc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:15.329013 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e0f4b2-17c0-47f8-8cd5-c5f46081dbfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:15.329646 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e0f4b2-17c0-47f8-8cd5-c5f46081dbfe None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:15.329921 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e0f4b2-17c0-47f8-8cd5-c5f46081dbfe None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:15.330189 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e0f4b2-17c0-47f8-8cd5-c5f46081dbfe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:15.330907 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e0f4b2-17c0-47f8-8cd5-c5f46081dbfe None admin] RBAC: Adding query filter params (group.id=0115efc9f6184a23bcf4d23ff19c2115, role.id=069fe3e033664493b3def844e3735bc9, scope.project.id=41f15a3cf54f4397920b0e8f758539ea) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:15.331215 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e0f4b2-17c0-47f8-8cd5-c5f46081dbfe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:15.332681 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e0f4b2-17c0-47f8-8cd5-c5f46081dbfe None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:15.345629 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 223/435] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Sep 8 01:00:15 2022] GET /identity/v3/role_assignments?group.id=0115efc9f6184a23bcf4d23ff19c2115&scope.project.id=41f15a3cf54f4397920b0e8f758539ea&role.id=069fe3e033664493b3def844e3735bc9 => generated 611 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:15.345629 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b204ee-6137-48ea-8c47-babac1abe7ae None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:15.345629 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b204ee-6137-48ea-8c47-babac1abe7ae None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:15.345629 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b204ee-6137-48ea-8c47-babac1abe7ae None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:15.345629 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 224/436] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:00:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:00:15.347777 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4655103-32aa-4180-b101-af701aca11e1 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:15.348120 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4655103-32aa-4180-b101-af701aca11e1 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:15.348412 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4655103-32aa-4180-b101-af701aca11e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:15.369127 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-e4655103-32aa-4180-b101-af701aca11e1 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:00:15.370349 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-e4655103-32aa-4180-b101-af701aca11e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:00:15.389339 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:15.406263 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:15.414385 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dc2a26-24ef-430e-879d-e87eef55ccc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:15.417613 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dc2a26-24ef-430e-879d-e87eef55ccc6 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:15.457493 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 225/437] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:00:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 01:00:15.550705 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 213/438] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 01:00:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 08 01:00:15.556873 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4655103-32aa-4180-b101-af701aca11e1 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:15.673692 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4655103-32aa-4180-b101-af701aca11e1 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:15.710515 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b81bf26-e75c-4f65-86c8-b6de40e80d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:15.711206 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81bf26-e75c-4f65-86c8-b6de40e80d7d None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:15.711480 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81bf26-e75c-4f65-86c8-b6de40e80d7d None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:15.711745 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81bf26-e75c-4f65-86c8-b6de40e80d7d None admin] PATH_INFO: `/v3/projects/0f19f058a73f49819b30c33d7fcfc52f` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:15.712910 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b81bf26-e75c-4f65-86c8-b6de40e80d7d None admin] RBAC: Authorizing `identity:get_project(project_id=0f19f058a73f49819b30c33d7fcfc52f)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:15.716236 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b81bf26-e75c-4f65-86c8-b6de40e80d7d None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:15.718262 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 226/439] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 01:00:15 2022] GET /identity/v3/projects/0f19f058a73f49819b30c33d7fcfc52f => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:15.724118 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc2a26-24ef-430e-879d-e87eef55ccc6 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:15.758357 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b81bf26-e75c-4f65-86c8-b6de40e80d7d None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:15.781063 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:15.836970 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:15.853445 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc2a26-24ef-430e-879d-e87eef55ccc6 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:15.883149 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b81bf26-e75c-4f65-86c8-b6de40e80d7d None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:15.890045 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e971c780-f138-4fed-860d-dc32dcf563db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:15.890701 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e971c780-f138-4fed-860d-dc32dcf563db None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:15.890979 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e971c780-f138-4fed-860d-dc32dcf563db None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:15.891248 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e971c780-f138-4fed-860d-dc32dcf563db None admin] PATH_INFO: `/v3/groups/0115efc9f6184a23bcf4d23ff19c2115` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:15.892364 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e971c780-f138-4fed-860d-dc32dcf563db None admin] RBAC: Authorizing `identity:get_group(group_id=0115efc9f6184a23bcf4d23ff19c2115)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:15.893805 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e971c780-f138-4fed-860d-dc32dcf563db None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:15.895351 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 214/440] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Sep 8 01:00:15 2022] GET /identity/v3/groups/0115efc9f6184a23bcf4d23ff19c2115 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:15.896469 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e393b-55a1-4901-be9d-84410b374566 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:15.896745 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e393b-55a1-4901-be9d-84410b374566 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:15.897012 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e393b-55a1-4901-be9d-84410b374566 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:15.900294 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 215/441] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:00:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:00:15.900852 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37e393b-55a1-4901-be9d-84410b374566 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:15.920653 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdc802a-9174-4bb3-94cd-f7de779ab261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:15.921278 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc802a-9174-4bb3-94cd-f7de779ab261 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:15.921555 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc802a-9174-4bb3-94cd-f7de779ab261 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:15.921825 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc802a-9174-4bb3-94cd-f7de779ab261 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:15.931709 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdc802a-9174-4bb3-94cd-f7de779ab261 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:00:15.936736 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-dbdc802a-9174-4bb3-94cd-f7de779ab261 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 01:00:15.936736 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:15.936736 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:15.936736 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:15.936736 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:15.936736 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:15.936736 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:15.936736 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:15.936736 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:15.936736 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:15.936736 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:15.936736 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:15.936736 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:00:15.936736 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:00:15.936736 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:00:15.936736 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:00:15.937809 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:00:15.937809 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:00:15.937809 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:00:15.937809 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:00:15.937809 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:15.937809 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:15.937809 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:15.937809 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:15.937809 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:15.937809 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:15.937809 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:15.937809 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:15.937809 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:15.937809 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:15.937809 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:15.937809 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:15.938616 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:15.938616 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:15.938616 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:15.938616 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:15.938616 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:00:15.938616 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:00:15.938616 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:00:15.938616 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:00:15.938616 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:00:15.938616 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:00:15.938616 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 01:00:15.938616 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:15.938616 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 227/442] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 8 01:00:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 01:00:15.939261 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc287ace-aac0-4940-8552-faff3ee26f27 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:15.939758 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc287ace-aac0-4940-8552-faff3ee26f27 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:15.939758 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc287ace-aac0-4940-8552-faff3ee26f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:15.957952 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-dc287ace-aac0-4940-8552-faff3ee26f27 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:00:15.959153 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-dc287ace-aac0-4940-8552-faff3ee26f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:00:16.003417 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37e393b-55a1-4901-be9d-84410b374566 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:16.025249 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 228/443] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:00:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 01:00:16.026066 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc287ace-aac0-4940-8552-faff3ee26f27 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:16.039206 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:16.042347 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0779ba23-d07c-41ad-91be-3ed801f24925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:16.043329 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0779ba23-d07c-41ad-91be-3ed801f24925 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:16.044321 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0779ba23-d07c-41ad-91be-3ed801f24925 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:16.044818 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0779ba23-d07c-41ad-91be-3ed801f24925 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:16.045587 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0779ba23-d07c-41ad-91be-3ed801f24925 None admin] RBAC: Adding query filter params (group.id=0115efc9f6184a23bcf4d23ff19c2115, role.id=92aa6d6824404d10b4a3be8b4a6bc317, scope.project.id=0f19f058a73f49819b30c33d7fcfc52f) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:16.045895 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0779ba23-d07c-41ad-91be-3ed801f24925 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:16.047347 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0779ba23-d07c-41ad-91be-3ed801f24925 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:16.053298 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 216/444] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Sep 8 01:00:15 2022] GET /identity/v3/role_assignments?group.id=0115efc9f6184a23bcf4d23ff19c2115&scope.project.id=0f19f058a73f49819b30c33d7fcfc52f&role.id=92aa6d6824404d10b4a3be8b4a6bc317 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:00:16.134534 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc287ace-aac0-4940-8552-faff3ee26f27 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:16.172067 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0470edda-7b4b-4f04-a918-37e6aacac88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:16.172769 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0470edda-7b4b-4f04-a918-37e6aacac88e None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:16.173009 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0470edda-7b4b-4f04-a918-37e6aacac88e None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:16.173284 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0470edda-7b4b-4f04-a918-37e6aacac88e None admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:16.173967 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0470edda-7b4b-4f04-a918-37e6aacac88e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:16.174272 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0470edda-7b4b-4f04-a918-37e6aacac88e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:16.175675 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0470edda-7b4b-4f04-a918-37e6aacac88e None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:16.185216 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 229/445] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 01:00:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:16.191006 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0779ba23-d07c-41ad-91be-3ed801f24925 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:16.211704 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:16.297340 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0470edda-7b4b-4f04-a918-37e6aacac88e None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:16.305998 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0779ba23-d07c-41ad-91be-3ed801f24925 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:16.343434 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c2f0f8-429e-451a-824b-ecdfa7e8ea3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:16.344488 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2f0f8-429e-451a-824b-ecdfa7e8ea3a None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:16.344855 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2f0f8-429e-451a-824b-ecdfa7e8ea3a None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:16.345145 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2f0f8-429e-451a-824b-ecdfa7e8ea3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:16.376513 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:16.413671 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0470edda-7b4b-4f04-a918-37e6aacac88e None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:16.422483 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c2f0f8-429e-451a-824b-ecdfa7e8ea3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:16.424027 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c2f0f8-429e-451a-824b-ecdfa7e8ea3a None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:16.443758 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:16.456763 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63caca52-ce14-4549-8358-6c3bb81b1cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:16.457396 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63caca52-ce14-4549-8358-6c3bb81b1cc8 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:16.457670 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63caca52-ce14-4549-8358-6c3bb81b1cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:16.457984 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63caca52-ce14-4549-8358-6c3bb81b1cc8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:16.468926 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63caca52-ce14-4549-8358-6c3bb81b1cc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:00:16.474096 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-63caca52-ce14-4549-8358-6c3bb81b1cc8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 01:00:16.474096 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:16.474096 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:16.474096 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:16.474096 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:16.474096 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:16.474096 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:16.474096 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:16.474096 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:16.474096 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:16.474096 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:16.474096 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:16.474096 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:00:16.474096 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:00:16.474096 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:00:16.474096 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:00:16.475100 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:00:16.475100 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:00:16.475100 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:00:16.475100 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:00:16.475100 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:16.475100 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:16.475100 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:16.475100 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:16.475100 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:16.475100 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:16.475100 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:16.475100 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:16.475100 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:16.475100 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:16.475100 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:16.475100 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:16.476061 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:16.476061 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:16.476061 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:16.476061 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:16.476061 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:00:16.476061 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:00:16.476061 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:00:16.476061 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:00:16.476061 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:00:16.476061 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:00:16.476061 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 01:00:16.476061 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:16.476061 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 230/446] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 8 01:00:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 01:00:16.480821 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63caca52-ce14-4549-8358-6c3bb81b1cc8 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:16.529476 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 217/447] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 01:00:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 01:00:16.535210 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c2f0f8-429e-451a-824b-ecdfa7e8ea3a None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:16.579427 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63caca52-ce14-4549-8358-6c3bb81b1cc8 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:16.607691 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:16.613956 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f190ae1-1d35-4df6-a930-f1a0126bb472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:16.614561 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f190ae1-1d35-4df6-a930-f1a0126bb472 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:16.614845 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f190ae1-1d35-4df6-a930-f1a0126bb472 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:16.615112 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f190ae1-1d35-4df6-a930-f1a0126bb472 None admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:16.615769 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f190ae1-1d35-4df6-a930-f1a0126bb472 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:16.616071 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f190ae1-1d35-4df6-a930-f1a0126bb472 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:16.617941 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f190ae1-1d35-4df6-a930-f1a0126bb472 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:16.628596 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 231/448] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 01:00:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:00:16.633777 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f190ae1-1d35-4df6-a930-f1a0126bb472 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:16.636624 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c2f0f8-429e-451a-824b-ecdfa7e8ea3a None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:16.654985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:16.672103 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6cd2d7-5564-4637-bf3b-7071d5847771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:16.672688 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6cd2d7-5564-4637-bf3b-7071d5847771 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:16.672962 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6cd2d7-5564-4637-bf3b-7071d5847771 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:16.673229 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6cd2d7-5564-4637-bf3b-7071d5847771 None admin] PATH_INFO: `/v3/projects/0f19f058a73f49819b30c33d7fcfc52f` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:16.674336 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6cd2d7-5564-4637-bf3b-7071d5847771 None admin] RBAC: Authorizing `identity:get_project(project_id=0f19f058a73f49819b30c33d7fcfc52f)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:16.675816 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6cd2d7-5564-4637-bf3b-7071d5847771 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:16.677434 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 218/449] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 01:00:16 2022] GET /identity/v3/projects/0f19f058a73f49819b30c33d7fcfc52f => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:00:16.682751 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6cd2d7-5564-4637-bf3b-7071d5847771 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:16.732525 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f190ae1-1d35-4df6-a930-f1a0126bb472 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:16.770007 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f40b8a-d56b-4474-a8a4-7ec1bd1bce60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:16.770609 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f40b8a-d56b-4474-a8a4-7ec1bd1bce60 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:16.770888 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f40b8a-d56b-4474-a8a4-7ec1bd1bce60 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:16.771156 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f40b8a-d56b-4474-a8a4-7ec1bd1bce60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:16.779827 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6cd2d7-5564-4637-bf3b-7071d5847771 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:16.814191 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494b9cf0-dbc5-4d6b-9d04-7ae90ec3874e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:16.814814 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b9cf0-dbc5-4d6b-9d04-7ae90ec3874e None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:16.815403 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b9cf0-dbc5-4d6b-9d04-7ae90ec3874e None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:16.815403 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b9cf0-dbc5-4d6b-9d04-7ae90ec3874e None admin] PATH_INFO: `/v3/groups/0115efc9f6184a23bcf4d23ff19c2115` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:16.816503 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494b9cf0-dbc5-4d6b-9d04-7ae90ec3874e None admin] RBAC: Authorizing `identity:get_group(group_id=0115efc9f6184a23bcf4d23ff19c2115)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:16.817970 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494b9cf0-dbc5-4d6b-9d04-7ae90ec3874e None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:16.819592 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 219/450] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Sep 8 01:00:16 2022] GET /identity/v3/groups/0115efc9f6184a23bcf4d23ff19c2115 => generated 226 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:00:16.825491 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494b9cf0-dbc5-4d6b-9d04-7ae90ec3874e None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:16.827127 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:16.836551 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f40b8a-d56b-4474-a8a4-7ec1bd1bce60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:16.838058 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f40b8a-d56b-4474-a8a4-7ec1bd1bce60 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:16.880608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:16.925327 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494b9cf0-dbc5-4d6b-9d04-7ae90ec3874e None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:16.936684 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 232/451] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 01:00:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 08 01:00:16.942400 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f40b8a-d56b-4474-a8a4-7ec1bd1bce60 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:16.957922 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddb2024-6910-48a9-bc04-2bdea4eba354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:16.958522 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb2024-6910-48a9-bc04-2bdea4eba354 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:16.958824 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb2024-6910-48a9-bc04-2bdea4eba354 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:16.959073 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb2024-6910-48a9-bc04-2bdea4eba354 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:16.959796 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddb2024-6910-48a9-bc04-2bdea4eba354 None admin] RBAC: Adding query filter params (group.id=0115efc9f6184a23bcf4d23ff19c2115, role.id=069fe3e033664493b3def844e3735bc9, scope.project.id=0f19f058a73f49819b30c33d7fcfc52f) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:16.960096 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddb2024-6910-48a9-bc04-2bdea4eba354 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:16.961555 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddb2024-6910-48a9-bc04-2bdea4eba354 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:16.968011 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 220/452] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Sep 8 01:00:16 2022] GET /identity/v3/role_assignments?group.id=0115efc9f6184a23bcf4d23ff19c2115&scope.project.id=0f19f058a73f49819b30c33d7fcfc52f&role.id=069fe3e033664493b3def844e3735bc9 => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:00:17.036110 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:17.045926 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f40b8a-d56b-4474-a8a4-7ec1bd1bce60 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:17.081487 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a22b439-e6bc-4ce1-8313-e76bf9efd8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:17.082128 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a22b439-e6bc-4ce1-8313-e76bf9efd8dd None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:17.082402 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a22b439-e6bc-4ce1-8313-e76bf9efd8dd None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:17.082672 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a22b439-e6bc-4ce1-8313-e76bf9efd8dd None admin] PATH_INFO: `/v3/projects/fc3cae79b39a4ab893f53897bb099478` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:17.083819 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a22b439-e6bc-4ce1-8313-e76bf9efd8dd None admin] RBAC: Authorizing `identity:get_project(project_id=fc3cae79b39a4ab893f53897bb099478)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:17.085323 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a22b439-e6bc-4ce1-8313-e76bf9efd8dd None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:17.086989 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 233/453] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 01:00:16 2022] GET /identity/v3/projects/fc3cae79b39a4ab893f53897bb099478 => generated 344 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:00:17.093126 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddb2024-6910-48a9-bc04-2bdea4eba354 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:17.197856 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddb2024-6910-48a9-bc04-2bdea4eba354 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:17.214513 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:17.232743 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8606e548-2b18-4ad1-80a1-a5822c133525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:17.233415 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8606e548-2b18-4ad1-80a1-a5822c133525 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:17.233693 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8606e548-2b18-4ad1-80a1-a5822c133525 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:17.233961 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8606e548-2b18-4ad1-80a1-a5822c133525 None admin] PATH_INFO: `/v3/groups/333dc8aa4d294de79d04753d2de626a2` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:17.235072 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8606e548-2b18-4ad1-80a1-a5822c133525 None admin] RBAC: Authorizing `identity:get_group(group_id=333dc8aa4d294de79d04753d2de626a2)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:17.236543 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8606e548-2b18-4ad1-80a1-a5822c133525 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:17.238260 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 221/454] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Sep 8 01:00:17 2022] GET /identity/v3/groups/333dc8aa4d294de79d04753d2de626a2 => generated 229 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:17.244063 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a22b439-e6bc-4ce1-8313-e76bf9efd8dd None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:17.345303 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a22b439-e6bc-4ce1-8313-e76bf9efd8dd None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:17.378791 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e4bf64-ef4b-4262-bddd-fce8b02c86bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:17.379389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4bf64-ef4b-4262-bddd-fce8b02c86bb None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:17.379667 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4bf64-ef4b-4262-bddd-fce8b02c86bb None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:17.379935 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4bf64-ef4b-4262-bddd-fce8b02c86bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:17.380683 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e4bf64-ef4b-4262-bddd-fce8b02c86bb None admin] RBAC: Adding query filter params (group.id=333dc8aa4d294de79d04753d2de626a2, role.id=34de369ec7c54715aa83a9052a26c3f2, scope.project.id=fc3cae79b39a4ab893f53897bb099478) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:17.380988 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e4bf64-ef4b-4262-bddd-fce8b02c86bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:17.382420 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e4bf64-ef4b-4262-bddd-fce8b02c86bb None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:17.388347 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 234/455] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Sep 8 01:00:17 2022] GET /identity/v3/role_assignments?group.id=333dc8aa4d294de79d04753d2de626a2&scope.project.id=fc3cae79b39a4ab893f53897bb099478&role.id=34de369ec7c54715aa83a9052a26c3f2 => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:19.968199 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c270c12-a500-4af6-9e78-a6ebb557df3e None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:19.968696 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c270c12-a500-4af6-9e78-a6ebb557df3e None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:19.968696 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c270c12-a500-4af6-9e78-a6ebb557df3e None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:19.970315 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 222/456] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:00:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:00:19.975340 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d72954-89f3-43ea-939d-924c9fc5d42b None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:19.975756 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d72954-89f3-43ea-939d-924c9fc5d42b None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:19.975878 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d72954-89f3-43ea-939d-924c9fc5d42b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:20.001187 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-e1d72954-89f3-43ea-939d-924c9fc5d42b None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:00:20.002402 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-e1d72954-89f3-43ea-939d-924c9fc5d42b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:00:20.027983 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:20.097801 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 235/457] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:00:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 01:00:20.321225 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8dda71-b075-47d6-87de-82d78e4794e7 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:20.321524 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8dda71-b075-47d6-87de-82d78e4794e7 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:20.321977 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8dda71-b075-47d6-87de-82d78e4794e7 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:20.323623 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 223/458] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:00:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:00:20.329348 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f15fb-1546-4210-9e38-6b3858fc5499 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:20.329635 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f15fb-1546-4210-9e38-6b3858fc5499 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:20.329911 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f15fb-1546-4210-9e38-6b3858fc5499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:20.341936 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b115c1-6bc9-48d8-93f7-99cdbebe26a1 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:20.342217 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b115c1-6bc9-48d8-93f7-99cdbebe26a1 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:20.342487 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b115c1-6bc9-48d8-93f7-99cdbebe26a1 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:20.344068 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 224/459] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:00:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:00:20.349329 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c4b21-24e3-4993-8b9d-e08bcbda0042 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:20.349895 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c4b21-24e3-4993-8b9d-e08bcbda0042 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:20.350373 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c4b21-24e3-4993-8b9d-e08bcbda0042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:20.352041 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-eb8f15fb-1546-4210-9e38-6b3858fc5499 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:00:20.353386 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-eb8f15fb-1546-4210-9e38-6b3858fc5499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:00:20.377600 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-533c4b21-24e3-4993-8b9d-e08bcbda0042 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:00:20.379311 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-533c4b21-24e3-4993-8b9d-e08bcbda0042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:00:20.463990 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 236/460] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:00:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 01:00:20.465009 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f15fb-1546-4210-9e38-6b3858fc5499 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:20.482528 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 225/461] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:00:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 01:00:20.573813 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:20.596079 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f15fb-1546-4210-9e38-6b3858fc5499 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:20.650235 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a83cb65-52cf-4279-8f7a-fbf18a00094b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:20.650934 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a83cb65-52cf-4279-8f7a-fbf18a00094b None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:20.651214 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a83cb65-52cf-4279-8f7a-fbf18a00094b None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:20.651546 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a83cb65-52cf-4279-8f7a-fbf18a00094b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:20.665822 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a83cb65-52cf-4279-8f7a-fbf18a00094b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:20.667275 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a83cb65-52cf-4279-8f7a-fbf18a00094b None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:20.688308 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-7a83cb65-52cf-4279-8f7a-fbf18a00094b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:20.688308 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:20.688308 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:20.688308 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:20.688308 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:20.688308 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:20.688308 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:20.688308 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:20.688308 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:20.688308 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:20.688308 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:20.688308 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:20.688308 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:00:20.688308 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:00:20.688308 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:00:20.688308 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:00:20.689446 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:20.689446 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:20.689446 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:20.689446 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:20.689446 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:20.689446 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:20.689446 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:20.689446 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:20.689446 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:20.689446 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:20.689446 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:20.689446 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:20.689446 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:20.689446 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:20.689446 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:20.689446 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:20.689446 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:00:20.690707 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:00:20.690707 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:20.690707 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:20.690707 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 237/462] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:00:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:20.695421 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533c4b21-24e3-4993-8b9d-e08bcbda0042 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:20.792876 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a83cb65-52cf-4279-8f7a-fbf18a00094b None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:20.819332 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533c4b21-24e3-4993-8b9d-e08bcbda0042 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:20.840073 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:20.873932 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d563c9-f011-4032-a184-1fba37e5b82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:20.874801 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d563c9-f011-4032-a184-1fba37e5b82b None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:20.875313 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d563c9-f011-4032-a184-1fba37e5b82b None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:20.875724 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d563c9-f011-4032-a184-1fba37e5b82b None admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:20.876617 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d563c9-f011-4032-a184-1fba37e5b82b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:20.877114 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d563c9-f011-4032-a184-1fba37e5b82b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:20.878803 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d563c9-f011-4032-a184-1fba37e5b82b None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:20.902667 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 226/463] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:00:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:20.903811 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d563c9-f011-4032-a184-1fba37e5b82b None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:20.939875 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a83cb65-52cf-4279-8f7a-fbf18a00094b None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:20.984971 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab636aa-8efd-4a5e-b78e-bd79ef787d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:20.985607 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab636aa-8efd-4a5e-b78e-bd79ef787d95 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:20.986038 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab636aa-8efd-4a5e-b78e-bd79ef787d95 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:20.986324 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab636aa-8efd-4a5e-b78e-bd79ef787d95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:20.989977 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:21.002096 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab636aa-8efd-4a5e-b78e-bd79ef787d95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:21.003838 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab636aa-8efd-4a5e-b78e-bd79ef787d95 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:21.021280 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d563c9-f011-4032-a184-1fba37e5b82b None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:21.022394 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-0ab636aa-8efd-4a5e-b78e-bd79ef787d95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:21.022394 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:21.022394 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:21.022394 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:21.022394 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:21.022394 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:21.022394 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:21.022394 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:21.022394 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:21.022394 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:21.022394 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:21.022394 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:21.022394 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:00:21.022394 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:00:21.022394 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:00:21.022394 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:00:21.023436 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:21.023436 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:21.023436 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:21.023436 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:21.023436 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:21.023436 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:21.023436 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:21.023436 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:21.023436 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:21.023436 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:21.023436 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:21.023436 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:21.023436 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:21.023436 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:21.023436 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:21.023436 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:21.023436 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:00:21.024344 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:00:21.024344 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:21.024344 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:21.024344 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 238/464] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:00:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 01:00:21.024565 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab636aa-8efd-4a5e-b78e-bd79ef787d95 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:21.069215 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8ece37-b51e-4813-836e-1c2480789496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:21.069849 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ece37-b51e-4813-836e-1c2480789496 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:21.070130 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ece37-b51e-4813-836e-1c2480789496 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:21.070403 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ece37-b51e-4813-836e-1c2480789496 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:21.082169 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8ece37-b51e-4813-836e-1c2480789496 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:21.083673 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8ece37-b51e-4813-836e-1c2480789496 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:21.100119 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-da8ece37-b51e-4813-836e-1c2480789496 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:21.100119 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:21.100119 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:21.100119 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:21.100119 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:21.100119 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:21.100119 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:21.100119 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:21.100119 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:21.100119 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:21.100119 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:21.100119 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:21.100119 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:00:21.100119 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:00:21.100119 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:00:21.100119 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:00:21.101032 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:21.101032 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:21.101032 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:21.101032 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:21.101032 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:21.101032 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:21.101032 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:21.101032 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:21.101032 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:21.101032 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:21.101032 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:21.101032 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:21.101032 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:21.101032 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:21.101032 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:21.101032 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:21.101032 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:00:21.101877 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:00:21.101877 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:21.101877 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:21.101877 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 227/465] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:00:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:21.102093 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8ece37-b51e-4813-836e-1c2480789496 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:21.139058 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:21.142074 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab636aa-8efd-4a5e-b78e-bd79ef787d95 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:21.179298 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afdd5d0-f9aa-4a02-8540-27a3d351ba57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:21.179908 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afdd5d0-f9aa-4a02-8540-27a3d351ba57 None admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:21.180728 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afdd5d0-f9aa-4a02-8540-27a3d351ba57 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:21.181011 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afdd5d0-f9aa-4a02-8540-27a3d351ba57 None admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:21.181674 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afdd5d0-f9aa-4a02-8540-27a3d351ba57 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:21.183133 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afdd5d0-f9aa-4a02-8540-27a3d351ba57 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:21.204907 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 239/466] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 8 01:00:21 2022] POST /identity/v3/users => generated 246 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:00:21.206144 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee72de37-2dd7-4f04-8845-2113e08286b3 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:21.206436 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee72de37-2dd7-4f04-8845-2113e08286b3 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:21.206710 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee72de37-2dd7-4f04-8845-2113e08286b3 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:21.212449 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 240/467] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:00:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:00:21.212449 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee72de37-2dd7-4f04-8845-2113e08286b3 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:21.243186 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8ece37-b51e-4813-836e-1c2480789496 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:21.283369 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:21.294827 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e7c56-0c1a-4156-a3dd-5f6415bc99d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:21.295533 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e7c56-0c1a-4156-a3dd-5f6415bc99d0 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:21.295884 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e7c56-0c1a-4156-a3dd-5f6415bc99d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:21.296367 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e7c56-0c1a-4156-a3dd-5f6415bc99d0 None admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:21.297210 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8e7c56-0c1a-4156-a3dd-5f6415bc99d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:21.297513 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8e7c56-0c1a-4156-a3dd-5f6415bc99d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:21.300430 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8e7c56-0c1a-4156-a3dd-5f6415bc99d0 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:21.319214 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 228/468] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:00:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:21.323983 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f7f05a-d6f1-405c-be6f-2907d6121be2 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:21.324342 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f7f05a-d6f1-405c-be6f-2907d6121be2 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:21.324699 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f7f05a-d6f1-405c-be6f-2907d6121be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:21.352012 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-64f7f05a-d6f1-405c-be6f-2907d6121be2 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:00:21.353846 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-64f7f05a-d6f1-405c-be6f-2907d6121be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:00:21.362512 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee72de37-2dd7-4f04-8845-2113e08286b3 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:21.404420 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e102e9-e930-47b3-b8f8-cb676193cd70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:21.405196 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e102e9-e930-47b3-b8f8-cb676193cd70 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:21.405478 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e102e9-e930-47b3-b8f8-cb676193cd70 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:21.405753 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e102e9-e930-47b3-b8f8-cb676193cd70 None admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:21.406402 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e102e9-e930-47b3-b8f8-cb676193cd70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:21.406706 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e102e9-e930-47b3-b8f8-cb676193cd70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:21.408164 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e102e9-e930-47b3-b8f8-cb676193cd70 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:21.422920 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 241/469] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:00:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:21.423737 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e102e9-e930-47b3-b8f8-cb676193cd70 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:21.446449 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 229/470] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:00:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 01:00:21.447348 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f7f05a-d6f1-405c-be6f-2907d6121be2 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:21.477871 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:21.558767 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e102e9-e930-47b3-b8f8-cb676193cd70 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:21.563204 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:21.587450 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f7f05a-d6f1-405c-be6f-2907d6121be2 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:21.610617 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29df2967-1b12-4328-b130-c0b9c8dde310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:21.611396 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df2967-1b12-4328-b130-c0b9c8dde310 None admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:21.611808 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df2967-1b12-4328-b130-c0b9c8dde310 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:21.612083 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df2967-1b12-4328-b130-c0b9c8dde310 None admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:21.613016 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29df2967-1b12-4328-b130-c0b9c8dde310 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:21.614869 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29df2967-1b12-4328-b130-c0b9c8dde310 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:21.646027 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7557e91-cab5-48d8-80b4-cef8d2498200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:21.646728 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7557e91-cab5-48d8-80b4-cef8d2498200 None admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:21.647013 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7557e91-cab5-48d8-80b4-cef8d2498200 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:21.647186 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 242/471] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 8 01:00:21 2022] POST /identity/v3/users => generated 248 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:00:21.647409 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7557e91-cab5-48d8-80b4-cef8d2498200 None admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:21.648260 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7557e91-cab5-48d8-80b4-cef8d2498200 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:21.648464 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aec481-c0df-4d49-9cd0-6a9f32d96ba1 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:21.648776 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aec481-c0df-4d49-9cd0-6a9f32d96ba1 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:21.649028 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aec481-c0df-4d49-9cd0-6a9f32d96ba1 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:21.650026 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7557e91-cab5-48d8-80b4-cef8d2498200 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:21.650226 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 243/472] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:00:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:00:21.656488 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bbb01b-0f7d-4e3e-a189-c082ef53135e None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:21.656767 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bbb01b-0f7d-4e3e-a189-c082ef53135e None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:21.657048 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bbb01b-0f7d-4e3e-a189-c082ef53135e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:21.681009 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 230/473] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 8 01:00:21 2022] POST /identity/v3/users => generated 248 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:00:21.681257 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-12bbb01b-0f7d-4e3e-a189-c082ef53135e None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:00:21.682563 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-12bbb01b-0f7d-4e3e-a189-c082ef53135e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:00:21.760310 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7557e91-cab5-48d8-80b4-cef8d2498200 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:21.765343 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 244/474] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:00:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 01:00:21.891522 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:21.927631 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7557e91-cab5-48d8-80b4-cef8d2498200 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:21.963604 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67effe7-a626-4b19-b575-3953c8b612ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:21.964579 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67effe7-a626-4b19-b575-3953c8b612ad None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:21.965041 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67effe7-a626-4b19-b575-3953c8b612ad None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:21.966167 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67effe7-a626-4b19-b575-3953c8b612ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:21.978903 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67effe7-a626-4b19-b575-3953c8b612ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:21.980680 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67effe7-a626-4b19-b575-3953c8b612ad None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:21.995889 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-d67effe7-a626-4b19-b575-3953c8b612ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:21.995889 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:21.995889 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:21.995889 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:21.995889 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:21.995889 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:21.995889 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:21.995889 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:21.995889 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:21.995889 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:21.995889 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:21.995889 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:21.995889 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:00:21.995889 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:00:21.995889 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:00:21.995889 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:00:21.997450 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:21.997450 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:21.997450 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:21.997450 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:21.997450 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:21.997450 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:21.997450 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:21.997450 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:21.997450 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:21.997450 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:21.997450 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:21.997450 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:21.997450 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:21.997450 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:21.997450 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:21.997450 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:21.997450 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:00:21.998271 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:00:21.998271 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:21.998271 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:22.000000 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 231/475] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:00:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:22.005632 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bbb01b-0f7d-4e3e-a189-c082ef53135e None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:22.084557 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67effe7-a626-4b19-b575-3953c8b612ad None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:22.117290 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:22.129467 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bbb01b-0f7d-4e3e-a189-c082ef53135e None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:22.180175 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4764450f-ccff-49c3-bd2f-09d303cfab6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:22.180836 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4764450f-ccff-49c3-bd2f-09d303cfab6d None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:22.181190 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4764450f-ccff-49c3-bd2f-09d303cfab6d None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:22.181540 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4764450f-ccff-49c3-bd2f-09d303cfab6d None admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:22.182272 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4764450f-ccff-49c3-bd2f-09d303cfab6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:22.182650 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4764450f-ccff-49c3-bd2f-09d303cfab6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:22.184700 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4764450f-ccff-49c3-bd2f-09d303cfab6d None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:22.200689 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 245/476] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:00:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:22.206063 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4764450f-ccff-49c3-bd2f-09d303cfab6d None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:22.242329 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67effe7-a626-4b19-b575-3953c8b612ad None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:22.314011 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:22.337165 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4764450f-ccff-49c3-bd2f-09d303cfab6d None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:22.344916 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6779daf-9c60-478a-9d8f-f7b0b3667e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:22.345542 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6779daf-9c60-478a-9d8f-f7b0b3667e4e None admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:22.345827 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6779daf-9c60-478a-9d8f-f7b0b3667e4e None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:22.346105 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6779daf-9c60-478a-9d8f-f7b0b3667e4e None admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:22.346809 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6779daf-9c60-478a-9d8f-f7b0b3667e4e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:22.348244 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6779daf-9c60-478a-9d8f-f7b0b3667e4e None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:22.372810 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.sql.core [None req-c6779daf-9c60-478a-9d8f-f7b0b3667e4e None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 08 01:00:22.372810 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 08 01:00:22.372810 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [parameters: {'id': '209226adf0e4491f84862ba6d00361fe', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 08 01:00:22.372810 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81661) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 08 01:00:22.376794 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c66a6f-0f2f-4221-871c-358c84466376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:22.377468 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c66a6f-0f2f-4221-871c-358c84466376 None admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:22.377753 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c66a6f-0f2f-4221-871c-358c84466376 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:22.378028 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c66a6f-0f2f-4221-871c-358c84466376 None admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:22.378729 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c66a6f-0f2f-4221-871c-358c84466376 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:22.380191 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c66a6f-0f2f-4221-871c-358c84466376 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:22.397744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-c6779daf-9c60-478a-9d8f-f7b0b3667e4e None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 08 01:00:22.397744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:22.397744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 08 01:00:22.397744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 08 01:00:22.397744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 08 01:00:22.397744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 08 01:00:22.397744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 08 01:00:22.397744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application result = self._query(query) Sep 08 01:00:22.397744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 08 01:00:22.397744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application conn.query(q) Sep 08 01:00:22.397744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 08 01:00:22.397744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 08 01:00:22.397744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 08 01:00:22.397744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application result.read() Sep 08 01:00:22.397744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 08 01:00:22.398666 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 08 01:00:22.398666 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 08 01:00:22.398666 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 08 01:00:22.398666 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 08 01:00:22.398666 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 08 01:00:22.398666 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 08 01:00:22.398666 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 08 01:00:22.398666 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 08 01:00:22.398666 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:22.398666 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 08 01:00:22.398666 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:22.398666 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:22.398666 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 08 01:00:22.398666 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 08 01:00:22.398666 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 08 01:00:22.398666 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return ref.to_dict() Sep 08 01:00:22.398666 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 08 01:00:22.398666 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application next(self.gen) Sep 08 01:00:22.399839 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 08 01:00:22.399839 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application yield resource Sep 08 01:00:22.399839 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 08 01:00:22.399839 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application next(self.gen) Sep 08 01:00:22.399839 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 08 01:00:22.399839 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application self.session.rollback() Sep 08 01:00:22.399839 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 08 01:00:22.399839 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application self.force_reraise() Sep 08 01:00:22.399839 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 08 01:00:22.399839 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise self.value Sep 08 01:00:22.399839 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 08 01:00:22.399839 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 08 01:00:22.399839 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 08 01:00:22.399839 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application session.commit() Sep 08 01:00:22.399839 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 08 01:00:22.399839 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 08 01:00:22.399839 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 08 01:00:22.401223 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application self._prepare_impl() Sep 08 01:00:22.401223 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 08 01:00:22.401223 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application self.session.flush() Sep 08 01:00:22.401223 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 08 01:00:22.401223 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application self._flush(objects) Sep 08 01:00:22.401223 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 08 01:00:22.401223 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 08 01:00:22.401223 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 08 01:00:22.401223 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application compat.raise_( Sep 08 01:00:22.401223 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 08 01:00:22.401223 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception Sep 08 01:00:22.401223 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 08 01:00:22.401223 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application flush_context.execute() Sep 08 01:00:22.401223 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 08 01:00:22.401223 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rec.execute(self) Sep 08 01:00:22.401223 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 08 01:00:22.401223 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 08 01:00:22.403135 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 08 01:00:22.403135 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 08 01:00:22.403135 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 08 01:00:22.403135 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 08 01:00:22.403135 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 08 01:00:22.403135 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 08 01:00:22.403135 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 08 01:00:22.403135 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 08 01:00:22.403135 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 08 01:00:22.403135 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 08 01:00:22.403135 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 08 01:00:22.403135 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 08 01:00:22.403135 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 08 01:00:22.403135 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 08 01:00:22.403135 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 08 01:00:22.403135 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception Sep 08 01:00:22.410090 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 08 01:00:22.410090 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 08 01:00:22.410090 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 08 01:00:22.410090 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 08 01:00:22.410090 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 08 01:00:22.410090 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application result = self._query(query) Sep 08 01:00:22.410090 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 08 01:00:22.410090 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application conn.query(q) Sep 08 01:00:22.410090 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 08 01:00:22.410090 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 08 01:00:22.410090 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 08 01:00:22.410090 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application result.read() Sep 08 01:00:22.410090 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 08 01:00:22.410090 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 08 01:00:22.410090 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 08 01:00:22.410090 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 08 01:00:22.412296 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 08 01:00:22.412296 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 08 01:00:22.412296 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 08 01:00:22.412296 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 08 01:00:22.412296 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 08 01:00:22.412296 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 08 01:00:22.412296 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [parameters: {'id': '209226adf0e4491f84862ba6d00361fe', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 08 01:00:22.412296 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 08 01:00:22.412296 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:22.412296 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 01:00:22.412296 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:22.412296 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:22.412296 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:22.412296 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:22.412296 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:22.412296 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:22.414547 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:22.414547 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:22.414547 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:22.414547 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:22.414547 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:22.414547 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:22.414547 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 08 01:00:22.414547 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 08 01:00:22.414547 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:22.414547 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:22.414547 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 08 01:00:22.414547 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 08 01:00:22.414547 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 08 01:00:22.414547 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 08 01:00:22.414547 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 08 01:00:22.414547 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:22.416706 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 232/477] 10.222.0.42 () {60 vars in 1131 bytes} [Thu Sep 8 01:00:22 2022] POST /identity/v3/roles => generated 144 bytes in 315 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 08 01:00:22.416706 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6779daf-9c60-478a-9d8f-f7b0b3667e4e None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:22.416977 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 246/478] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 8 01:00:22 2022] POST /identity/v3/users => generated 249 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:00:22.529573 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6779daf-9c60-478a-9d8f-f7b0b3667e4e None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:22.567493 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c1f410-daf0-441f-9b25-aff0387926f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:22.568157 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c1f410-daf0-441f-9b25-aff0387926f0 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:22.568516 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c1f410-daf0-441f-9b25-aff0387926f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:22.568801 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c1f410-daf0-441f-9b25-aff0387926f0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:22.581481 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c1f410-daf0-441f-9b25-aff0387926f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:00:22.586979 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-27c1f410-daf0-441f-9b25-aff0387926f0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 01:00:22.586979 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:22.586979 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:22.586979 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:22.586979 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:22.586979 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:22.586979 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:22.586979 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:22.586979 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:22.586979 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:22.586979 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:22.586979 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:22.586979 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:00:22.586979 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:00:22.586979 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:00:22.586979 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:00:22.589044 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:00:22.589044 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:00:22.589044 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:00:22.589044 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:00:22.589044 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:22.589044 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:22.589044 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:22.589044 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:22.589044 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:22.589044 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:22.589044 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:22.589044 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:22.589044 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:22.589044 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:22.589044 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:22.589044 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:22.590123 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:22.590123 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:22.590123 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:22.590123 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:22.590123 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:00:22.590123 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:00:22.590123 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:00:22.590123 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:00:22.590123 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:00:22.590123 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:00:22.590123 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 01:00:22.590123 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:22.590123 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 233/479] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 8 01:00:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:22.594432 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c66a6f-0f2f-4221-871c-358c84466376 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:22.604954 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:22.713180 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c66a6f-0f2f-4221-871c-358c84466376 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:22.751176 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50be12af-c5af-4ab6-845d-935331c6cb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:22.751809 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be12af-c5af-4ab6-845d-935331c6cb17 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:22.752090 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be12af-c5af-4ab6-845d-935331c6cb17 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:22.752404 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be12af-c5af-4ab6-845d-935331c6cb17 None admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:22.753093 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50be12af-c5af-4ab6-845d-935331c6cb17 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:22.753398 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50be12af-c5af-4ab6-845d-935331c6cb17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:22.754811 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50be12af-c5af-4ab6-845d-935331c6cb17 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:22.765041 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 247/480] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 01:00:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:00:24.191781 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e0ac4-9a40-4e73-97ef-f5fdc0a7bed4 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:24.192505 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e0ac4-9a40-4e73-97ef-f5fdc0a7bed4 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:24.192505 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e0ac4-9a40-4e73-97ef-f5fdc0a7bed4 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:24.193979 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 234/481] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:00:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:00:24.198949 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9928168-286e-44d0-8974-d1a8e5fb15a5 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:24.199522 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9928168-286e-44d0-8974-d1a8e5fb15a5 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:24.199795 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9928168-286e-44d0-8974-d1a8e5fb15a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:24.219064 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-a9928168-286e-44d0-8974-d1a8e5fb15a5 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:00:24.224414 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-a9928168-286e-44d0-8974-d1a8e5fb15a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:00:24.305407 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:24.318126 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 248/482] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:00:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 01:00:24.434959 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19f34b-89f2-4bea-9d55-2b4001047d14 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:24.435303 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19f34b-89f2-4bea-9d55-2b4001047d14 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:24.435500 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19f34b-89f2-4bea-9d55-2b4001047d14 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:24.437149 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 235/483] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:00:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:00:24.442901 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040281f4-f575-495f-ba12-6a15e27f63ac None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:24.443183 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040281f4-f575-495f-ba12-6a15e27f63ac None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:24.443370 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040281f4-f575-495f-ba12-6a15e27f63ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:24.469690 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-040281f4-f575-495f-ba12-6a15e27f63ac None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:00:24.470932 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-040281f4-f575-495f-ba12-6a15e27f63ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:00:24.562627 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 249/484] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:00:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 01:00:24.582179 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef59f62-0a1d-4046-8f0f-d751ab5a715a None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:24.582469 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef59f62-0a1d-4046-8f0f-d751ab5a715a None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:24.582737 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef59f62-0a1d-4046-8f0f-d751ab5a715a None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:24.584227 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 236/485] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:00:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:00:24.589512 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78983b7-730f-4f0a-af2b-00f8ab9e3a29 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:24.590037 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78983b7-730f-4f0a-af2b-00f8ab9e3a29 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:24.590570 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78983b7-730f-4f0a-af2b-00f8ab9e3a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:24.609883 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-e78983b7-730f-4f0a-af2b-00f8ab9e3a29 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:00:24.611344 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-e78983b7-730f-4f0a-af2b-00f8ab9e3a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:00:24.647621 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef59f62-0a1d-4046-8f0f-d751ab5a715a None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:24.686364 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 250/486] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:00:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 01:00:24.731827 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:24.797829 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef59f62-0a1d-4046-8f0f-d751ab5a715a None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:24.841167 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070635da-e3a1-42c8-a1a2-574f2ea393f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:24.841809 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070635da-e3a1-42c8-a1a2-574f2ea393f0 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:24.842092 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070635da-e3a1-42c8-a1a2-574f2ea393f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:24.842364 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070635da-e3a1-42c8-a1a2-574f2ea393f0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:24.858318 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070635da-e3a1-42c8-a1a2-574f2ea393f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:00:24.863176 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-070635da-e3a1-42c8-a1a2-574f2ea393f0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 01:00:24.863176 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:24.863176 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:24.863176 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:24.863176 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:24.863176 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:24.863176 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:24.863176 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:24.863176 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:24.863176 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:24.863176 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:24.863176 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:24.863176 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:00:24.863176 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:00:24.863176 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:00:24.863176 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:00:24.864053 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:00:24.864053 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:00:24.864053 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:00:24.864053 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:00:24.864053 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:24.864053 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:24.864053 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:24.864053 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:24.864053 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:24.864053 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:24.864053 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:24.864053 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:24.864053 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:24.864053 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:24.864053 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:24.864053 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:24.864869 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:24.864869 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:24.864869 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:24.864869 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:24.864869 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:00:24.864869 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:00:24.864869 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:00:24.864869 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:00:24.864869 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:00:24.864869 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:00:24.864869 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 01:00:24.864869 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:24.864869 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 237/487] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 8 01:00:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:24.870282 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78983b7-730f-4f0a-af2b-00f8ab9e3a29 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:24.947721 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:25.000776 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070635da-e3a1-42c8-a1a2-574f2ea393f0 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:25.005539 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78983b7-730f-4f0a-af2b-00f8ab9e3a29 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:25.058487 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348bb6eb-e32e-4692-bcf8-1c5fb81917a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:25.059630 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348bb6eb-e32e-4692-bcf8-1c5fb81917a1 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:25.059914 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348bb6eb-e32e-4692-bcf8-1c5fb81917a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:25.060213 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348bb6eb-e32e-4692-bcf8-1c5fb81917a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:25.060888 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348bb6eb-e32e-4692-bcf8-1c5fb81917a1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:25.061192 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348bb6eb-e32e-4692-bcf8-1c5fb81917a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:25.062578 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348bb6eb-e32e-4692-bcf8-1c5fb81917a1 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:25.072409 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 251/488] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 8 01:00:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:25.073406 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348bb6eb-e32e-4692-bcf8-1c5fb81917a1 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:25.120351 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070635da-e3a1-42c8-a1a2-574f2ea393f0 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:25.132366 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:25.173956 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe94ae1d-eb05-42cd-b42a-4de5af8b72d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:25.174903 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94ae1d-eb05-42cd-b42a-4de5af8b72d4 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:25.175200 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94ae1d-eb05-42cd-b42a-4de5af8b72d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:25.175483 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94ae1d-eb05-42cd-b42a-4de5af8b72d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:25.191439 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe94ae1d-eb05-42cd-b42a-4de5af8b72d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:00:25.198013 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348bb6eb-e32e-4692-bcf8-1c5fb81917a1 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:25.198632 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-fe94ae1d-eb05-42cd-b42a-4de5af8b72d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 01:00:25.198632 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:25.198632 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:25.198632 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:25.198632 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:25.198632 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:25.198632 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:25.198632 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:25.198632 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:25.198632 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:25.198632 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:25.198632 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:25.198632 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:00:25.198632 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:00:25.198632 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:00:25.198632 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:00:25.199862 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:00:25.199862 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:00:25.199862 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:00:25.199862 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:00:25.199862 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:25.199862 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:25.199862 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:25.199862 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:25.199862 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:25.199862 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:25.199862 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:25.199862 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:25.199862 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:25.199862 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:25.199862 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:25.199862 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:25.200923 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:25.200923 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:25.200923 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:25.200923 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:25.200923 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:00:25.200923 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:00:25.200923 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:00:25.200923 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:00:25.200923 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:00:25.200923 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:00:25.200923 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 01:00:25.200923 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:25.200923 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 238/489] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 8 01:00:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 01:00:25.204762 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe94ae1d-eb05-42cd-b42a-4de5af8b72d4 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:25.225682 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:25.242106 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854b81d1-ed9d-4611-869f-c11e442f8e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:25.243037 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854b81d1-ed9d-4611-869f-c11e442f8e8b None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:25.243535 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854b81d1-ed9d-4611-869f-c11e442f8e8b None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:25.243984 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854b81d1-ed9d-4611-869f-c11e442f8e8b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:25.261338 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854b81d1-ed9d-4611-869f-c11e442f8e8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:00:25.269859 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-854b81d1-ed9d-4611-869f-c11e442f8e8b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 01:00:25.269859 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:25.269859 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:25.269859 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:25.269859 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:25.269859 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:25.269859 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:25.269859 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:25.269859 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:25.269859 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:25.269859 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:25.269859 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:25.269859 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:00:25.269859 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:00:25.269859 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:00:25.269859 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:00:25.270780 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:00:25.270780 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:00:25.270780 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:00:25.270780 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:00:25.270780 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:25.270780 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:25.270780 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:25.270780 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:25.270780 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:25.270780 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:25.270780 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:25.270780 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:25.270780 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:25.270780 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:25.270780 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:25.270780 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:25.271793 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:25.271793 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:25.271793 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:25.271793 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:25.271793 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:00:25.271793 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:00:25.271793 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:00:25.271793 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:00:25.271793 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:00:25.271793 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:00:25.271793 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 01:00:25.271793 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:25.271793 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 252/490] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 8 01:00:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:25.271793 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854b81d1-ed9d-4611-869f-c11e442f8e8b None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:25.331678 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe94ae1d-eb05-42cd-b42a-4de5af8b72d4 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:25.372389 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589302b8-7aee-4e39-9bb7-3667f5ed0846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:25.373273 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589302b8-7aee-4e39-9bb7-3667f5ed0846 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:25.373578 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589302b8-7aee-4e39-9bb7-3667f5ed0846 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:25.373953 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589302b8-7aee-4e39-9bb7-3667f5ed0846 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:25.386357 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589302b8-7aee-4e39-9bb7-3667f5ed0846 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:25.388911 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589302b8-7aee-4e39-9bb7-3667f5ed0846 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:25.400162 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854b81d1-ed9d-4611-869f-c11e442f8e8b None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:25.404658 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-589302b8-7aee-4e39-9bb7-3667f5ed0846 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:25.404658 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:25.404658 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:25.404658 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:25.404658 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:25.404658 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:25.404658 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:25.404658 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:25.404658 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:25.404658 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:25.404658 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:25.404658 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:25.404658 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:00:25.404658 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:00:25.404658 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:00:25.404658 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:00:25.405742 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:25.405742 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:25.405742 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:25.405742 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:25.405742 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:25.405742 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:25.405742 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:25.405742 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:25.405742 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:25.405742 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:25.405742 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:25.405742 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:25.405742 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:25.405742 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:25.405742 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:25.405742 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:25.405742 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:00:25.406812 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:00:25.406812 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:25.406812 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:25.408199 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 239/491] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:00:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:25.409068 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589302b8-7aee-4e39-9bb7-3667f5ed0846 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:25.433891 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:25.444602 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1322cc4-7488-40e1-8d52-ff4b7e08114e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:25.445587 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1322cc4-7488-40e1-8d52-ff4b7e08114e None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:25.446001 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1322cc4-7488-40e1-8d52-ff4b7e08114e None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:25.446439 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1322cc4-7488-40e1-8d52-ff4b7e08114e None admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:25.447300 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1322cc4-7488-40e1-8d52-ff4b7e08114e None admin] RBAC: Adding query filter params (name=service) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:25.447782 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1322cc4-7488-40e1-8d52-ff4b7e08114e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:25.449347 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1322cc4-7488-40e1-8d52-ff4b7e08114e None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:25.459792 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 253/492] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 8 01:00:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:25.461060 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1322cc4-7488-40e1-8d52-ff4b7e08114e None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:25.507780 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:25.513977 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589302b8-7aee-4e39-9bb7-3667f5ed0846 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:25.546969 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880b0c67-7cb5-407f-a710-2984ff5fbad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:25.547570 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b0c67-7cb5-407f-a710-2984ff5fbad4 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:25.547851 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b0c67-7cb5-407f-a710-2984ff5fbad4 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:25.548120 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b0c67-7cb5-407f-a710-2984ff5fbad4 None admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:25.548802 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880b0c67-7cb5-407f-a710-2984ff5fbad4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:25.549103 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880b0c67-7cb5-407f-a710-2984ff5fbad4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:25.550491 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880b0c67-7cb5-407f-a710-2984ff5fbad4 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:25.558073 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 240/493] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 8 01:00:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:25.558884 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880b0c67-7cb5-407f-a710-2984ff5fbad4 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:25.569544 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1322cc4-7488-40e1-8d52-ff4b7e08114e None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:25.603550 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e8ee9d-242f-4841-8886-ed4ad8867481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:25.604176 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8ee9d-242f-4841-8886-ed4ad8867481 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:25.604461 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8ee9d-242f-4841-8886-ed4ad8867481 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:25.604733 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8ee9d-242f-4841-8886-ed4ad8867481 None admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:25.605372 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e8ee9d-242f-4841-8886-ed4ad8867481 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:25.605692 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e8ee9d-242f-4841-8886-ed4ad8867481 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:25.607088 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e8ee9d-242f-4841-8886-ed4ad8867481 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:25.621272 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 254/494] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:00:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:25.623338 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c889d86a-6169-49e8-8e10-f7519a2d1e19 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:25.623616 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c889d86a-6169-49e8-8e10-f7519a2d1e19 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:25.623880 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c889d86a-6169-49e8-8e10-f7519a2d1e19 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:25.625095 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 255/495] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:00:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:00:25.626083 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a292ede-373f-45f3-825e-2587bceaf1af None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:25.626430 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a292ede-373f-45f3-825e-2587bceaf1af None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:25.626704 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a292ede-373f-45f3-825e-2587bceaf1af None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:25.628157 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 256/496] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:00:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:00:25.631432 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a292ede-373f-45f3-825e-2587bceaf1af None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:25.649074 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:25.664633 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880b0c67-7cb5-407f-a710-2984ff5fbad4 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:25.698799 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ac28a-da96-469e-b153-20a2bdd34a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:25.699441 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ac28a-da96-469e-b153-20a2bdd34a35 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:25.699774 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ac28a-da96-469e-b153-20a2bdd34a35 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:25.700117 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ac28a-da96-469e-b153-20a2bdd34a35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:25.710293 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955ac28a-da96-469e-b153-20a2bdd34a35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:25.711773 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955ac28a-da96-469e-b153-20a2bdd34a35 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:25.726618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-955ac28a-da96-469e-b153-20a2bdd34a35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:25.726618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:25.726618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:25.726618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:25.726618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:25.726618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:25.726618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:25.726618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:25.726618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:25.726618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:25.726618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:25.726618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:25.726618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:00:25.726618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:00:25.726618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:00:25.726618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:00:25.727618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:25.727618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:25.727618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:25.727618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:25.727618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:25.727618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:25.727618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:25.727618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:25.727618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:25.727618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:25.727618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:25.727618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:25.727618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:25.727618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:25.727618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:25.727618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:25.727618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:00:25.728585 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:00:25.728585 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:25.728585 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:25.728585 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 241/497] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:00:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:25.729029 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ac28a-da96-469e-b153-20a2bdd34a35 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:25.739065 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a292ede-373f-45f3-825e-2587bceaf1af None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:25.740638 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:25.775586 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f37aae-d085-44e5-a7f7-ef07ec4c1540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:25.776166 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f37aae-d085-44e5-a7f7-ef07ec4c1540 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:25.776433 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f37aae-d085-44e5-a7f7-ef07ec4c1540 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:25.776703 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f37aae-d085-44e5-a7f7-ef07ec4c1540 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:25.786615 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f37aae-d085-44e5-a7f7-ef07ec4c1540 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:25.788053 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f37aae-d085-44e5-a7f7-ef07ec4c1540 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:25.802401 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-96f37aae-d085-44e5-a7f7-ef07ec4c1540 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:25.802401 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:25.802401 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:25.802401 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:25.802401 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:25.802401 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:25.802401 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:25.802401 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:25.802401 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:25.802401 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:25.802401 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:25.802401 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:25.802401 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:00:25.802401 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:00:25.802401 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:00:25.802401 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:00:25.803318 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:25.803318 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:25.803318 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:25.803318 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:25.803318 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:25.803318 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:25.803318 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:25.803318 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:25.803318 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:25.803318 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:25.803318 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:25.803318 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:25.803318 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:25.803318 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:25.803318 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:25.803318 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:25.803318 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:00:25.804190 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:00:25.804190 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:25.804190 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:25.804190 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 257/498] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:00:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:25.805099 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4315eb36-514d-4534-a20f-c7a56c330b94 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:25.805676 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4315eb36-514d-4534-a20f-c7a56c330b94 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:25.805676 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4315eb36-514d-4534-a20f-c7a56c330b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:25.823441 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-4315eb36-514d-4534-a20f-c7a56c330b94 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:00:25.825212 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-4315eb36-514d-4534-a20f-c7a56c330b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:00:25.831173 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ac28a-da96-469e-b153-20a2bdd34a35 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:25.864408 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdce9b9-55b4-4590-967f-320e0705bb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:25.865028 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdce9b9-55b4-4590-967f-320e0705bb75 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:25.865307 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdce9b9-55b4-4590-967f-320e0705bb75 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:25.865578 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdce9b9-55b4-4590-967f-320e0705bb75 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:25.869336 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:25.874901 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdce9b9-55b4-4590-967f-320e0705bb75 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:25.876680 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdce9b9-55b4-4590-967f-320e0705bb75 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:25.889347 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-2bdce9b9-55b4-4590-967f-320e0705bb75 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 01:00:25.889347 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:25.889347 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:25.889347 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:25.889347 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:25.889347 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:25.889347 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:25.889347 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:25.889347 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:25.889347 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:25.889347 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:25.889347 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:25.889347 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 01:00:25.889347 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 01:00:25.889347 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 01:00:25.889347 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 01:00:25.890231 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:25.890231 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:25.890231 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 01:00:25.890231 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:00:25.890231 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 01:00:25.890231 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:00:25.890231 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:25.890231 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:25.890231 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:25.890231 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:25.890231 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:25.890231 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:25.890231 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:25.890231 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:25.890231 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:25.890231 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:25.890231 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:25.890231 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:25.891125 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:25.891125 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:25.891125 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 01:00:25.891125 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 01:00:25.891125 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 01:00:25.891125 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 01:00:25.891125 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 01:00:25.891125 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 01:00:25.891125 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 01:00:25.891125 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:25.891125 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 242/499] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Sep 8 01:00:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:25.891682 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 258/500] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:00:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 01:00:25.892244 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497aab66-01da-450d-be71-c2b55dbfd98f None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:25.892417 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497aab66-01da-450d-be71-c2b55dbfd98f None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:25.892559 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4315eb36-514d-4534-a20f-c7a56c330b94 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:25.892702 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497aab66-01da-450d-be71-c2b55dbfd98f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:25.917090 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-497aab66-01da-450d-be71-c2b55dbfd98f None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:00:25.918929 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-497aab66-01da-450d-be71-c2b55dbfd98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:00:25.978286 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:25.997738 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4315eb36-514d-4534-a20f-c7a56c330b94 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:25.998543 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 243/501] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:00:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 01:00:25.999380 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497aab66-01da-450d-be71-c2b55dbfd98f None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:26.041963 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9965b6e1-db04-45f0-9a0d-4ff6fac12b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:26.042879 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9965b6e1-db04-45f0-9a0d-4ff6fac12b2a None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:26.043456 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9965b6e1-db04-45f0-9a0d-4ff6fac12b2a None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:26.043930 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9965b6e1-db04-45f0-9a0d-4ff6fac12b2a None admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:26.044813 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9965b6e1-db04-45f0-9a0d-4ff6fac12b2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:26.045315 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9965b6e1-db04-45f0-9a0d-4ff6fac12b2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:26.047634 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9965b6e1-db04-45f0-9a0d-4ff6fac12b2a None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:26.061493 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 259/502] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:00:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:26.063364 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9965b6e1-db04-45f0-9a0d-4ff6fac12b2a None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:26.102967 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497aab66-01da-450d-be71-c2b55dbfd98f None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:26.130253 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:26.140814 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd2e75-caf2-4b65-9ea5-7013b428aac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:26.141543 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd2e75-caf2-4b65-9ea5-7013b428aac6 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:26.141896 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd2e75-caf2-4b65-9ea5-7013b428aac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:26.142238 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd2e75-caf2-4b65-9ea5-7013b428aac6 None admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:26.143003 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cd2e75-caf2-4b65-9ea5-7013b428aac6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:26.143384 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cd2e75-caf2-4b65-9ea5-7013b428aac6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:26.144883 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cd2e75-caf2-4b65-9ea5-7013b428aac6 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:26.156761 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 244/503] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:00:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:26.162200 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd2e75-caf2-4b65-9ea5-7013b428aac6 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:26.188359 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9965b6e1-db04-45f0-9a0d-4ff6fac12b2a None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:26.225123 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1fd157-7896-4aef-a094-69bb6a744482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:26.226046 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1fd157-7896-4aef-a094-69bb6a744482 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:26.226575 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1fd157-7896-4aef-a094-69bb6a744482 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:26.226575 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1fd157-7896-4aef-a094-69bb6a744482 None admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:26.227304 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1fd157-7896-4aef-a094-69bb6a744482 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:26.227608 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1fd157-7896-4aef-a094-69bb6a744482 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:26.229239 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1fd157-7896-4aef-a094-69bb6a744482 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:26.271478 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 260/504] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Sep 8 01:00:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:26.271478 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1fd157-7896-4aef-a094-69bb6a744482 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:26.271478 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:26.275254 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd2e75-caf2-4b65-9ea5-7013b428aac6 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:26.308881 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd2d7ef-068c-477b-82e3-cd59c7674b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:26.309433 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2d7ef-068c-477b-82e3-cd59c7674b51 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:26.309705 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2d7ef-068c-477b-82e3-cd59c7674b51 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:26.309976 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2d7ef-068c-477b-82e3-cd59c7674b51 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:26.317375 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd2d7ef-068c-477b-82e3-cd59c7674b51 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:26.318830 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd2d7ef-068c-477b-82e3-cd59c7674b51 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:26.330640 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-0fd2d7ef-068c-477b-82e3-cd59c7674b51 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 08 01:00:26.330640 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:26.330640 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:26.330640 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:26.330640 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:26.330640 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:26.330640 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:26.330640 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:26.330640 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:26.330640 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:26.330640 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:26.330640 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:26.330640 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 01:00:26.330640 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 01:00:26.330640 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 01:00:26.330640 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 01:00:26.331709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:26.331709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:26.331709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 01:00:26.331709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:00:26.331709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 01:00:26.331709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:00:26.331709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:26.331709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:26.331709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:26.331709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:26.331709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:26.331709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:26.331709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:26.331709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:26.331709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:26.331709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:26.331709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:26.331709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:26.332809 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:26.332809 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:26.332809 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 01:00:26.332809 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 01:00:26.332809 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 01:00:26.332809 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 01:00:26.332809 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 01:00:26.332809 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 01:00:26.332809 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 08 01:00:26.332809 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:26.332809 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 245/505] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 01:00:26 2022] GET /identity/v3/users/glance => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:26.333466 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd2d7ef-068c-477b-82e3-cd59c7674b51 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:26.372190 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1fd157-7896-4aef-a094-69bb6a744482 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:26.379062 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:26.406696 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2a511a-88a9-420e-8cc7-7dd36bdf601b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:26.407323 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a511a-88a9-420e-8cc7-7dd36bdf601b None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:26.407605 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a511a-88a9-420e-8cc7-7dd36bdf601b None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:26.408021 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a511a-88a9-420e-8cc7-7dd36bdf601b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:26.414950 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2a511a-88a9-420e-8cc7-7dd36bdf601b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:26.416436 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2a511a-88a9-420e-8cc7-7dd36bdf601b None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:26.429147 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-7b2a511a-88a9-420e-8cc7-7dd36bdf601b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 01:00:26.429147 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:26.429147 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:26.429147 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:26.429147 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:26.429147 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:26.429147 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:26.429147 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:26.429147 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:26.429147 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:26.429147 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:26.429147 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:26.429147 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 01:00:26.429147 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 01:00:26.429147 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 01:00:26.429147 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 01:00:26.430153 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:26.430153 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:26.430153 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 01:00:26.430153 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:00:26.430153 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 01:00:26.430153 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:00:26.430153 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:26.430153 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:26.430153 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:26.430153 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:26.430153 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:26.430153 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:26.430153 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:26.430153 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:26.430153 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:26.430153 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:26.430153 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:26.430153 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:26.431402 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:26.431402 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:26.431402 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 01:00:26.431402 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 01:00:26.431402 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 01:00:26.431402 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 01:00:26.431402 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 01:00:26.431402 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 01:00:26.431402 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 01:00:26.431402 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:26.431402 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 261/506] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 01:00:26 2022] GET /identity/v3/users/cinder => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:26.431402 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2a511a-88a9-420e-8cc7-7dd36bdf601b None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:26.438564 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd2d7ef-068c-477b-82e3-cd59c7674b51 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:26.472318 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefe2737-a0cf-4a11-a545-524b8fd41a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:26.472948 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe2737-a0cf-4a11-a545-524b8fd41a55 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:26.473227 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe2737-a0cf-4a11-a545-524b8fd41a55 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:26.473498 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe2737-a0cf-4a11-a545-524b8fd41a55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:26.483987 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefe2737-a0cf-4a11-a545-524b8fd41a55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:26.485504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefe2737-a0cf-4a11-a545-524b8fd41a55 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:26.493092 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:26.500312 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-eefe2737-a0cf-4a11-a545-524b8fd41a55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:26.500312 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:26.500312 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:26.500312 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:26.500312 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:26.500312 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:26.500312 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:26.500312 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:26.500312 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:26.500312 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:26.500312 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:26.500312 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:26.500312 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:00:26.500312 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:00:26.500312 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:00:26.500312 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:00:26.501400 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:26.501400 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:26.501400 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:26.501400 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:26.501400 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:26.501400 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:26.501400 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:26.501400 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:26.501400 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:26.501400 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:26.501400 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:26.501400 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:26.501400 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:26.501400 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:26.501400 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:26.501400 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:26.501400 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:00:26.502318 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:00:26.502318 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:26.502318 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:26.502318 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 246/507] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:00:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:26.502621 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefe2737-a0cf-4a11-a545-524b8fd41a55 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:26.536514 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2a511a-88a9-420e-8cc7-7dd36bdf601b None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:26.569178 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c15eaf4-40ac-47ae-8e25-244ecddbf752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:26.569711 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c15eaf4-40ac-47ae-8e25-244ecddbf752 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:26.569989 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c15eaf4-40ac-47ae-8e25-244ecddbf752 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:26.570258 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c15eaf4-40ac-47ae-8e25-244ecddbf752 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:26.598606 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:26.601909 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefe2737-a0cf-4a11-a545-524b8fd41a55 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:26.635545 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c011c-6ea2-4842-bbfc-be39bf56d082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:26.636468 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c011c-6ea2-4842-bbfc-be39bf56d082 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:26.636740 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c011c-6ea2-4842-bbfc-be39bf56d082 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:26.637681 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c011c-6ea2-4842-bbfc-be39bf56d082 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:26.640046 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c15eaf4-40ac-47ae-8e25-244ecddbf752 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:26.641588 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c15eaf4-40ac-47ae-8e25-244ecddbf752 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:26.647616 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0c011c-6ea2-4842-bbfc-be39bf56d082 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:00:26.652888 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-5e0c011c-6ea2-4842-bbfc-be39bf56d082 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 01:00:26.652888 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:26.652888 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:26.652888 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:26.652888 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:26.652888 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:26.652888 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:26.652888 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:26.652888 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:26.652888 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:26.652888 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:26.652888 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:26.652888 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:00:26.652888 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:00:26.652888 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:00:26.652888 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:00:26.654046 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:00:26.654046 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:00:26.654046 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:00:26.654046 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:00:26.654046 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:26.654046 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:26.654046 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:26.654046 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:26.654046 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:26.654046 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:26.654046 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:26.654046 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:26.654046 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:26.654046 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:26.654046 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:26.654046 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:26.654843 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:26.654843 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:26.654843 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:26.654843 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:26.654843 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:00:26.654843 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:00:26.654843 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:00:26.654843 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:00:26.654843 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:00:26.654843 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:00:26.654843 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 01:00:26.654843 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:26.654843 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 247/508] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 8 01:00:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:26.656851 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c011c-6ea2-4842-bbfc-be39bf56d082 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:26.700741 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:26.740791 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 262/509] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 01:00:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 01:00:26.741724 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c15eaf4-40ac-47ae-8e25-244ecddbf752 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:26.759769 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c011c-6ea2-4842-bbfc-be39bf56d082 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:26.794494 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2998792-2822-4fe9-a0ba-f127f401f0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:26.795090 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2998792-2822-4fe9-a0ba-f127f401f0bb None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:26.795375 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2998792-2822-4fe9-a0ba-f127f401f0bb None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:26.795645 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2998792-2822-4fe9-a0ba-f127f401f0bb None admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:26.796397 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2998792-2822-4fe9-a0ba-f127f401f0bb None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:26.796702 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2998792-2822-4fe9-a0ba-f127f401f0bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:26.798167 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2998792-2822-4fe9-a0ba-f127f401f0bb None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:26.834915 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 248/510] 10.222.0.42 () {56 vars in 1137 bytes} [Thu Sep 8 01:00:26 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:26.835668 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2998792-2822-4fe9-a0ba-f127f401f0bb None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:26.838072 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c15eaf4-40ac-47ae-8e25-244ecddbf752 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:26.853642 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:26.872681 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ec44f-ae95-4b1e-9b50-281ff2bc9aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:26.873286 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ec44f-ae95-4b1e-9b50-281ff2bc9aaa None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:26.873567 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ec44f-ae95-4b1e-9b50-281ff2bc9aaa None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:26.873840 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ec44f-ae95-4b1e-9b50-281ff2bc9aaa None admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:26.874567 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ec44f-ae95-4b1e-9b50-281ff2bc9aaa None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:26.874870 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ec44f-ae95-4b1e-9b50-281ff2bc9aaa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:26.876373 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ec44f-ae95-4b1e-9b50-281ff2bc9aaa None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:26.895242 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 263/511] 10.222.0.42 () {56 vars in 1137 bytes} [Thu Sep 8 01:00:26 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:26.896033 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ec44f-ae95-4b1e-9b50-281ff2bc9aaa None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:26.916694 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:26.940676 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2998792-2822-4fe9-a0ba-f127f401f0bb None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:26.974505 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2560b4f-a16f-46cb-a48c-aceb79a6afed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:26.975103 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2560b4f-a16f-46cb-a48c-aceb79a6afed None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:26.975385 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2560b4f-a16f-46cb-a48c-aceb79a6afed None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:26.975659 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2560b4f-a16f-46cb-a48c-aceb79a6afed None admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:26.976331 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2560b4f-a16f-46cb-a48c-aceb79a6afed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:26.976643 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2560b4f-a16f-46cb-a48c-aceb79a6afed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:26.978033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2560b4f-a16f-46cb-a48c-aceb79a6afed None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:26.993999 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 249/512] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:00:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:26.993999 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2560b4f-a16f-46cb-a48c-aceb79a6afed None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:26.995007 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ec44f-ae95-4b1e-9b50-281ff2bc9aaa None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:27.029877 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98cfa21-a6c9-40f3-89a8-dcba35b09508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:27.030473 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98cfa21-a6c9-40f3-89a8-dcba35b09508 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:27.030751 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98cfa21-a6c9-40f3-89a8-dcba35b09508 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:27.031021 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98cfa21-a6c9-40f3-89a8-dcba35b09508 None admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:27.031674 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98cfa21-a6c9-40f3-89a8-dcba35b09508 None admin] RBAC: Adding query filter params (name=service) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:27.031974 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98cfa21-a6c9-40f3-89a8-dcba35b09508 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:27.033390 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98cfa21-a6c9-40f3-89a8-dcba35b09508 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:27.046029 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 264/513] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 8 01:00:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:27.046029 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98cfa21-a6c9-40f3-89a8-dcba35b09508 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:27.068441 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:27.092907 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2560b4f-a16f-46cb-a48c-aceb79a6afed None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:27.122572 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:27.126986 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4561b0-42d0-4d0a-80e7-a5bcefd30dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:27.127610 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4561b0-42d0-4d0a-80e7-a5bcefd30dfa None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:27.127892 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4561b0-42d0-4d0a-80e7-a5bcefd30dfa None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:27.128184 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4561b0-42d0-4d0a-80e7-a5bcefd30dfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:27.138944 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4561b0-42d0-4d0a-80e7-a5bcefd30dfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:27.140437 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4561b0-42d0-4d0a-80e7-a5bcefd30dfa None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:27.140956 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98cfa21-a6c9-40f3-89a8-dcba35b09508 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:27.154560 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-7d4561b0-42d0-4d0a-80e7-a5bcefd30dfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:27.154560 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:27.154560 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:27.154560 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:27.154560 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:27.154560 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:27.154560 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:27.154560 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:27.154560 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:27.154560 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:27.154560 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:27.154560 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:27.154560 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:00:27.154560 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:00:27.154560 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:00:27.154560 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:00:27.155619 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:27.155619 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:27.155619 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:27.155619 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:27.155619 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:27.155619 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:27.155619 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:27.155619 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:27.155619 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:27.155619 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:27.155619 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:27.155619 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:27.155619 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:27.155619 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:27.155619 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:27.155619 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:27.155619 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:00:27.156640 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:00:27.156640 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:27.156640 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:27.156640 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 250/514] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:00:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:27.156640 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4561b0-42d0-4d0a-80e7-a5bcefd30dfa None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:27.179796 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3592bb1e-c4f5-4088-a341-00b9eafa75fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:27.180430 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592bb1e-c4f5-4088-a341-00b9eafa75fe None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:27.180711 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592bb1e-c4f5-4088-a341-00b9eafa75fe None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:27.180982 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592bb1e-c4f5-4088-a341-00b9eafa75fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:27.247467 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3592bb1e-c4f5-4088-a341-00b9eafa75fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:27.248942 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3592bb1e-c4f5-4088-a341-00b9eafa75fe None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:27.254405 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4561b0-42d0-4d0a-80e7-a5bcefd30dfa None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:27.280174 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:27.288064 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6610b92e-5757-4a71-8cc3-010666f4e1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:27.288689 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610b92e-5757-4a71-8cc3-010666f4e1e9 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:27.288967 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610b92e-5757-4a71-8cc3-010666f4e1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:27.289238 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610b92e-5757-4a71-8cc3-010666f4e1e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:27.330388 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:27.350106 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 265/515] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 01:00:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 01:00:27.351061 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3592bb1e-c4f5-4088-a341-00b9eafa75fe None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:27.358485 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6610b92e-5757-4a71-8cc3-010666f4e1e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:27.359971 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6610b92e-5757-4a71-8cc3-010666f4e1e9 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:27.449551 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3592bb1e-c4f5-4088-a341-00b9eafa75fe None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:27.457136 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 251/516] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 01:00:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 01:00:27.457978 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6610b92e-5757-4a71-8cc3-010666f4e1e9 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:27.486086 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:27.493627 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc99f365-671e-4494-ab41-e26ad217f143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:27.494235 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99f365-671e-4494-ab41-e26ad217f143 None admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:27.494512 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99f365-671e-4494-ab41-e26ad217f143 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:27.494789 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99f365-671e-4494-ab41-e26ad217f143 None admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:27.495440 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc99f365-671e-4494-ab41-e26ad217f143 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:27.497167 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc99f365-671e-4494-ab41-e26ad217f143 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:27.518623 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 266/517] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 8 01:00:27 2022] POST /identity/v3/users => generated 247 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:00:27.519434 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc99f365-671e-4494-ab41-e26ad217f143 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:27.569018 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:27.586005 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6610b92e-5757-4a71-8cc3-010666f4e1e9 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:27.633260 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c015eb-0d32-4027-bb7a-0fdf334fcd39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:27.633885 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c015eb-0d32-4027-bb7a-0fdf334fcd39 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:27.634334 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c015eb-0d32-4027-bb7a-0fdf334fcd39 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:27.634607 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c015eb-0d32-4027-bb7a-0fdf334fcd39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:27.645432 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc99f365-671e-4494-ab41-e26ad217f143 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:27.646862 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c015eb-0d32-4027-bb7a-0fdf334fcd39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:27.648901 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c015eb-0d32-4027-bb7a-0fdf334fcd39 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:27.666360 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-95c015eb-0d32-4027-bb7a-0fdf334fcd39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:27.666360 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:27.666360 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:27.666360 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:27.666360 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:27.666360 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:27.666360 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:27.666360 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:27.666360 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:27.666360 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:27.666360 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:27.666360 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:27.666360 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:00:27.666360 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:00:27.666360 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:00:27.666360 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:00:27.667365 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:27.667365 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:27.667365 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:27.667365 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:27.667365 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:27.667365 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:27.667365 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:27.667365 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:27.667365 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:27.667365 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:27.667365 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:27.667365 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:27.667365 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:27.667365 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:27.667365 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:27.667365 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:27.667365 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:00:27.668771 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:00:27.668771 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:27.668771 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:27.668771 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 252/518] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:00:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:27.669098 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c015eb-0d32-4027-bb7a-0fdf334fcd39 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:27.694682 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf9026-c3ca-4098-9ab3-eaf40dbdc8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:27.695296 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf9026-c3ca-4098-9ab3-eaf40dbdc8d2 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:27.695573 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf9026-c3ca-4098-9ab3-eaf40dbdc8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:27.695847 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf9026-c3ca-4098-9ab3-eaf40dbdc8d2 None admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:27.696964 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf9026-c3ca-4098-9ab3-eaf40dbdc8d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:27.697279 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf9026-c3ca-4098-9ab3-eaf40dbdc8d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:27.698681 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf9026-c3ca-4098-9ab3-eaf40dbdc8d2 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:27.711947 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 267/519] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:00:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:27.712560 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf9026-c3ca-4098-9ab3-eaf40dbdc8d2 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:27.769857 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:27.791204 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c015eb-0d32-4027-bb7a-0fdf334fcd39 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:27.821158 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:27.825980 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf9026-c3ca-4098-9ab3-eaf40dbdc8d2 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:27.830331 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e3d7b2-213b-41a2-b9fa-627a9be9949b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:27.831162 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e3d7b2-213b-41a2-b9fa-627a9be9949b None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:27.831582 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e3d7b2-213b-41a2-b9fa-627a9be9949b None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:27.832001 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e3d7b2-213b-41a2-b9fa-627a9be9949b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:27.845779 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e3d7b2-213b-41a2-b9fa-627a9be9949b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:27.847842 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e3d7b2-213b-41a2-b9fa-627a9be9949b None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:27.862526 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3a6e8a-3a3e-4f42-a21d-0ada86b0091b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:27.863159 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-60e3d7b2-213b-41a2-b9fa-627a9be9949b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:27.863159 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:27.863159 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:27.863159 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:27.863159 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:27.863159 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:27.863159 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:27.863159 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:27.863159 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:27.863159 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:27.863159 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:27.863159 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:27.863159 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:00:27.863159 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:00:27.863159 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:00:27.863159 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:00:27.864007 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:27.864007 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:27.864007 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:27.864007 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:27.864007 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:27.864007 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:27.864007 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:27.864007 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:27.864007 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:27.864007 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:27.864007 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:27.864007 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:27.864007 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:27.864007 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:27.864007 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:27.864007 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:27.864007 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:00:27.864868 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:00:27.864868 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:27.864868 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:27.865025 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a6e8a-3a3e-4f42-a21d-0ada86b0091b None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:27.865025 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a6e8a-3a3e-4f42-a21d-0ada86b0091b None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:27.865025 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a6e8a-3a3e-4f42-a21d-0ada86b0091b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:27.865201 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 253/520] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:00:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:27.865201 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e3d7b2-213b-41a2-b9fa-627a9be9949b None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:27.876187 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3a6e8a-3a3e-4f42-a21d-0ada86b0091b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:27.877660 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3a6e8a-3a3e-4f42-a21d-0ada86b0091b None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:27.893628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-4f3a6e8a-3a3e-4f42-a21d-0ada86b0091b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:27.893628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:27.893628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:27.893628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:27.893628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:27.893628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:27.893628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:27.893628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:27.893628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:27.893628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:27.893628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:27.893628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:27.893628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:00:27.893628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:00:27.893628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:00:27.893628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:00:27.896253 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:27.896253 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:27.896253 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:27.896253 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:27.896253 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:27.896253 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:27.896253 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:27.896253 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:27.896253 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:27.896253 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:27.896253 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:27.896253 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:27.896253 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:27.896253 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:27.896253 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:27.896253 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:27.896253 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:00:27.897118 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:00:27.897118 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:27.897118 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:27.897118 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 268/521] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:00:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:27.897118 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3a6e8a-3a3e-4f42-a21d-0ada86b0091b None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:27.982485 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e3d7b2-213b-41a2-b9fa-627a9be9949b None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:28.010341 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3a6e8a-3a3e-4f42-a21d-0ada86b0091b None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:28.013476 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:28.019003 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a064bdab-7a22-4d1d-9b3a-76dee3dee5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:28.019697 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a064bdab-7a22-4d1d-9b3a-76dee3dee5ed None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:28.020041 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a064bdab-7a22-4d1d-9b3a-76dee3dee5ed None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:28.020398 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a064bdab-7a22-4d1d-9b3a-76dee3dee5ed None admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:28.021047 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a064bdab-7a22-4d1d-9b3a-76dee3dee5ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:28.021342 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a064bdab-7a22-4d1d-9b3a-76dee3dee5ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:28.022727 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a064bdab-7a22-4d1d-9b3a-76dee3dee5ed None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:28.035716 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 254/522] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:00:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:28.040691 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a064bdab-7a22-4d1d-9b3a-76dee3dee5ed None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:28.049579 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58474b-f1d1-4d18-a930-d5aaf3b45376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:28.050218 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58474b-f1d1-4d18-a930-d5aaf3b45376 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:28.050499 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58474b-f1d1-4d18-a930-d5aaf3b45376 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:28.050772 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58474b-f1d1-4d18-a930-d5aaf3b45376 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:28.057605 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d58474b-f1d1-4d18-a930-d5aaf3b45376 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:28.060094 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d58474b-f1d1-4d18-a930-d5aaf3b45376 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:28.072649 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-9d58474b-f1d1-4d18-a930-d5aaf3b45376 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:00:28.072649 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:28.072649 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:28.072649 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:28.072649 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:28.072649 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:28.072649 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:28.072649 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:28.072649 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:28.072649 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:28.072649 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:28.072649 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:28.072649 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 01:00:28.072649 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 01:00:28.072649 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 01:00:28.073504 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 01:00:28.073504 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:28.073504 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:28.073504 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:28.073504 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:28.073504 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:28.073504 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:28.073504 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:28.073504 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:28.073504 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:28.073504 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:28.073504 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:28.073504 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:28.073504 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:28.073504 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:28.073504 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:28.073504 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:28.074414 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 01:00:28.074414 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 01:00:28.074414 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 01:00:28.074414 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 01:00:28.074414 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 01:00:28.074414 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 01:00:28.074414 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:00:28.074414 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:28.079151 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 269/523] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 01:00:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:28.079972 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58474b-f1d1-4d18-a930-d5aaf3b45376 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:28.093450 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:28.146143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a064bdab-7a22-4d1d-9b3a-76dee3dee5ed None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:28.184259 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209cc47d-77b4-425d-8c7f-8ffbc7080eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:28.185204 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209cc47d-77b4-425d-8c7f-8ffbc7080eed None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:28.185809 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209cc47d-77b4-425d-8c7f-8ffbc7080eed None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:28.186091 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209cc47d-77b4-425d-8c7f-8ffbc7080eed None admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:28.187001 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209cc47d-77b4-425d-8c7f-8ffbc7080eed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:28.187307 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209cc47d-77b4-425d-8c7f-8ffbc7080eed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:28.189449 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209cc47d-77b4-425d-8c7f-8ffbc7080eed None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:28.193904 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58474b-f1d1-4d18-a930-d5aaf3b45376 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:28.207855 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 255/524] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:00:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:28.207855 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209cc47d-77b4-425d-8c7f-8ffbc7080eed None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:28.228558 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b6b690-a0bb-4f92-83d5-5d6d5ba7b802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:28.229165 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6b690-a0bb-4f92-83d5-5d6d5ba7b802 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:28.229449 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6b690-a0bb-4f92-83d5-5d6d5ba7b802 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:28.229722 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6b690-a0bb-4f92-83d5-5d6d5ba7b802 None admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:28.230345 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b6b690-a0bb-4f92-83d5-5d6d5ba7b802 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:28.230977 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b6b690-a0bb-4f92-83d5-5d6d5ba7b802 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:28.232392 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b6b690-a0bb-4f92-83d5-5d6d5ba7b802 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:28.244151 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 270/525] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:00:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:28.245088 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b6b690-a0bb-4f92-83d5-5d6d5ba7b802 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:28.275852 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:28.320495 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:28.326802 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209cc47d-77b4-425d-8c7f-8ffbc7080eed None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:28.358008 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b6b690-a0bb-4f92-83d5-5d6d5ba7b802 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:28.365163 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3009916-187b-4e08-80a8-b93d82e74d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:28.366217 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3009916-187b-4e08-80a8-b93d82e74d34 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:28.366510 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3009916-187b-4e08-80a8-b93d82e74d34 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:28.366779 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3009916-187b-4e08-80a8-b93d82e74d34 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:28.376902 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3009916-187b-4e08-80a8-b93d82e74d34 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:28.378365 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3009916-187b-4e08-80a8-b93d82e74d34 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:28.393232 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-b3009916-187b-4e08-80a8-b93d82e74d34 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 08 01:00:28.393232 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:28.393232 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:28.393232 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:28.393232 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:28.393232 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:28.393232 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:28.393232 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:28.393232 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:28.393232 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:28.393232 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:28.393232 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:28.393232 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 01:00:28.393232 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 01:00:28.393232 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 01:00:28.393232 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 01:00:28.394131 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:28.394131 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:28.394131 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 01:00:28.394131 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:00:28.394131 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 01:00:28.394131 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:00:28.394131 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:28.394131 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:28.394131 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:28.394131 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:28.394131 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:28.394131 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:28.394131 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:28.394131 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:28.394131 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:28.394131 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:28.394131 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:28.394131 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:28.400059 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:28.400059 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:28.400059 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 01:00:28.400059 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 01:00:28.400059 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 01:00:28.400059 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 01:00:28.400059 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 01:00:28.400059 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 01:00:28.400059 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 08 01:00:28.400059 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:28.400605 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf06f8-bf5e-48a5-ba6f-4351479857a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:28.400605 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf06f8-bf5e-48a5-ba6f-4351479857a6 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:28.400605 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf06f8-bf5e-48a5-ba6f-4351479857a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:28.400605 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf06f8-bf5e-48a5-ba6f-4351479857a6 None admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:28.400940 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 256/526] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 8 01:00:28 2022] GET /identity/v3/users/neutron => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:28.401006 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bf06f8-bf5e-48a5-ba6f-4351479857a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:28.401006 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bf06f8-bf5e-48a5-ba6f-4351479857a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:28.401120 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3009916-187b-4e08-80a8-b93d82e74d34 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:28.401192 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bf06f8-bf5e-48a5-ba6f-4351479857a6 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:28.416893 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 271/527] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 8 01:00:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:28.418806 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf06f8-bf5e-48a5-ba6f-4351479857a6 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:28.514791 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3009916-187b-4e08-80a8-b93d82e74d34 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:28.527735 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf06f8-bf5e-48a5-ba6f-4351479857a6 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:28.527981 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:28.552012 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee49263-4191-4eb9-9a86-ec19e3591c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:28.552884 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee49263-4191-4eb9-9a86-ec19e3591c67 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:28.553167 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee49263-4191-4eb9-9a86-ec19e3591c67 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:28.553438 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee49263-4191-4eb9-9a86-ec19e3591c67 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:28.559971 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:28.560882 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee49263-4191-4eb9-9a86-ec19e3591c67 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:28.562291 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee49263-4191-4eb9-9a86-ec19e3591c67 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:28.566835 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61a8d29-acb9-4122-bdde-82b7517361c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:28.567587 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61a8d29-acb9-4122-bdde-82b7517361c2 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:28.567966 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61a8d29-acb9-4122-bdde-82b7517361c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:28.568367 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61a8d29-acb9-4122-bdde-82b7517361c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:28.574415 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-9ee49263-4191-4eb9-9a86-ec19e3591c67 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:00:28.574415 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:28.574415 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:28.574415 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:28.574415 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:28.574415 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:28.574415 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:28.574415 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:28.574415 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:28.574415 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:28.574415 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:28.574415 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:28.574415 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 01:00:28.574415 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 01:00:28.574415 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 01:00:28.575536 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 01:00:28.575536 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:28.575536 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:28.575536 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:28.575536 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:28.575536 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:28.575536 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:28.575536 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:28.575536 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:28.575536 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:28.575536 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:28.575536 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:28.575536 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:28.575536 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:28.575536 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:28.575536 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:28.575536 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:28.576618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 01:00:28.576618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 01:00:28.576618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 01:00:28.576618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 01:00:28.576618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 01:00:28.576618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 01:00:28.576618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:00:28.576618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:28.577892 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61a8d29-acb9-4122-bdde-82b7517361c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:28.579432 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61a8d29-acb9-4122-bdde-82b7517361c2 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:28.580318 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 257/528] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 01:00:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:28.581131 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee49263-4191-4eb9-9a86-ec19e3591c67 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:28.593733 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-f61a8d29-acb9-4122-bdde-82b7517361c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:00:28.593733 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:28.593733 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:28.593733 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:28.593733 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:28.593733 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:28.593733 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:28.593733 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:28.593733 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:28.593733 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:28.593733 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:28.593733 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:28.593733 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 01:00:28.593733 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 01:00:28.593733 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 01:00:28.594732 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 01:00:28.594732 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:28.594732 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:28.594732 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:28.594732 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:28.594732 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:28.594732 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:28.594732 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:28.594732 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:28.594732 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:28.594732 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:28.594732 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:28.594732 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:28.594732 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:28.594732 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:28.594732 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:28.594732 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:28.600560 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 01:00:28.600560 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 01:00:28.600560 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 01:00:28.600560 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 01:00:28.600560 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 01:00:28.600560 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 01:00:28.600560 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:00:28.600560 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:28.600560 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 272/529] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 01:00:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:28.600560 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61a8d29-acb9-4122-bdde-82b7517361c2 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:28.695101 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee49263-4191-4eb9-9a86-ec19e3591c67 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:28.707512 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61a8d29-acb9-4122-bdde-82b7517361c2 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:28.730762 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde2bb52-5b66-44fa-b33d-05bfcf65c276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:28.731711 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2bb52-5b66-44fa-b33d-05bfcf65c276 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:28.731980 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2bb52-5b66-44fa-b33d-05bfcf65c276 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:28.732277 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2bb52-5b66-44fa-b33d-05bfcf65c276 None admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:28.733002 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde2bb52-5b66-44fa-b33d-05bfcf65c276 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:28.733304 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde2bb52-5b66-44fa-b33d-05bfcf65c276 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:28.735231 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde2bb52-5b66-44fa-b33d-05bfcf65c276 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:28.749847 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2229f90a-fb33-4be3-ad7b-0a2752026457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:28.750460 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2229f90a-fb33-4be3-ad7b-0a2752026457 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:28.750738 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2229f90a-fb33-4be3-ad7b-0a2752026457 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:28.751010 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2229f90a-fb33-4be3-ad7b-0a2752026457 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:28.751749 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2229f90a-fb33-4be3-ad7b-0a2752026457 None admin] RBAC: Adding query filter params (role.id=609d73f56e6a48698397131168abe6df, scope.project.id=78ad2a8fb5264fb4af935e039d28c8de, user.id=ae643cfdcb164aa9b474747cd21aa116) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:28.752061 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2229f90a-fb33-4be3-ad7b-0a2752026457 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:28.753550 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2229f90a-fb33-4be3-ad7b-0a2752026457 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:28.759469 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 258/530] 10.222.0.42 () {56 vars in 1139 bytes} [Thu Sep 8 01:00:28 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:28.760418 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde2bb52-5b66-44fa-b33d-05bfcf65c276 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:28.764659 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 273/531] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 8 01:00:28 2022] GET /identity/v3/role_assignments?user.id=ae643cfdcb164aa9b474747cd21aa116&scope.project.id=78ad2a8fb5264fb4af935e039d28c8de&role.id=609d73f56e6a48698397131168abe6df => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:28.765517 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2229f90a-fb33-4be3-ad7b-0a2752026457 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:28.829960 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:28.887264 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:28.982449 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2229f90a-fb33-4be3-ad7b-0a2752026457 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:28.989498 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde2bb52-5b66-44fa-b33d-05bfcf65c276 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:29.024335 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f925481-62c4-4be7-8d63-f9022ef9b295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:29.024946 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f925481-62c4-4be7-8d63-f9022ef9b295 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:29.025222 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f925481-62c4-4be7-8d63-f9022ef9b295 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:29.025495 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f925481-62c4-4be7-8d63-f9022ef9b295 None admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:29.026194 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f925481-62c4-4be7-8d63-f9022ef9b295 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:29.026500 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f925481-62c4-4be7-8d63-f9022ef9b295 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:29.028168 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f925481-62c4-4be7-8d63-f9022ef9b295 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:29.045898 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6d03e4-9b9a-4993-a796-3c017d40d6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:29.046514 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d03e4-9b9a-4993-a796-3c017d40d6c6 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:29.046793 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d03e4-9b9a-4993-a796-3c017d40d6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:29.047064 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d03e4-9b9a-4993-a796-3c017d40d6c6 None admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:29.047761 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6d03e4-9b9a-4993-a796-3c017d40d6c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:29.048069 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6d03e4-9b9a-4993-a796-3c017d40d6c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:29.049568 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6d03e4-9b9a-4993-a796-3c017d40d6c6 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:29.065916 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 274/532] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 8 01:00:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:29.065916 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f925481-62c4-4be7-8d63-f9022ef9b295 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:29.089705 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 259/533] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 8 01:00:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:29.090672 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6d03e4-9b9a-4993-a796-3c017d40d6c6 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:29.231512 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f925481-62c4-4be7-8d63-f9022ef9b295 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:29.246025 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:29.250275 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:29.263296 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6d03e4-9b9a-4993-a796-3c017d40d6c6 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:29.272065 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac95f4-7ba1-4f34-a3da-3c72fe1e69f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:29.272704 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac95f4-7ba1-4f34-a3da-3c72fe1e69f9 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:29.272982 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac95f4-7ba1-4f34-a3da-3c72fe1e69f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:29.273253 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac95f4-7ba1-4f34-a3da-3c72fe1e69f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:29.303240 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579a7d1c-c717-4f68-a2f7-3b077fe045d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:29.304294 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a7d1c-c717-4f68-a2f7-3b077fe045d4 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:29.304851 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a7d1c-c717-4f68-a2f7-3b077fe045d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:29.305325 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a7d1c-c717-4f68-a2f7-3b077fe045d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:29.306451 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579a7d1c-c717-4f68-a2f7-3b077fe045d4 None admin] RBAC: Adding query filter params (role.id=609d73f56e6a48698397131168abe6df, scope.project.id=78ad2a8fb5264fb4af935e039d28c8de, user.id=a307e9150ead4877b970e09f36553968) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:29.306978 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579a7d1c-c717-4f68-a2f7-3b077fe045d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:29.309754 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579a7d1c-c717-4f68-a2f7-3b077fe045d4 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:29.316174 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 260/534] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 8 01:00:29 2022] GET /identity/v3/role_assignments?user.id=a307e9150ead4877b970e09f36553968&scope.project.id=78ad2a8fb5264fb4af935e039d28c8de&role.id=609d73f56e6a48698397131168abe6df => generated 259 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:29.317309 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579a7d1c-c717-4f68-a2f7-3b077fe045d4 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:29.380589 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ac95f4-7ba1-4f34-a3da-3c72fe1e69f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:29.384441 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ac95f4-7ba1-4f34-a3da-3c72fe1e69f9 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:29.521882 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 275/535] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 01:00:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 01:00:29.528027 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac95f4-7ba1-4f34-a3da-3c72fe1e69f9 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:29.550060 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:29.595254 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579a7d1c-c717-4f68-a2f7-3b077fe045d4 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:29.663990 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac95f4-7ba1-4f34-a3da-3c72fe1e69f9 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:29.701830 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a34e66-8899-42db-a5f8-d778184692f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:29.702456 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a34e66-8899-42db-a5f8-d778184692f6 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:29.702733 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a34e66-8899-42db-a5f8-d778184692f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:29.703004 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a34e66-8899-42db-a5f8-d778184692f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:29.703787 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a34e66-8899-42db-a5f8-d778184692f6 None admin] RBAC: Adding query filter params (role.id=609d73f56e6a48698397131168abe6df, scope.project.id=78ad2a8fb5264fb4af935e039d28c8de, user.id=d1acfa7a09d34e2a9045c3f3fbea0f52) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:29.704047 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a34e66-8899-42db-a5f8-d778184692f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:29.705567 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a34e66-8899-42db-a5f8-d778184692f6 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:29.706882 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfab6996-8a7f-4804-b370-ddb28817aa8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:29.707820 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab6996-8a7f-4804-b370-ddb28817aa8e None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:29.708386 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab6996-8a7f-4804-b370-ddb28817aa8e None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:29.708962 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab6996-8a7f-4804-b370-ddb28817aa8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:29.720207 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 261/536] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 8 01:00:29 2022] GET /identity/v3/role_assignments?user.id=d1acfa7a09d34e2a9045c3f3fbea0f52&scope.project.id=78ad2a8fb5264fb4af935e039d28c8de&role.id=609d73f56e6a48698397131168abe6df => generated 259 bytes in 403 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:29.721674 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfab6996-8a7f-4804-b370-ddb28817aa8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:29.723429 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfab6996-8a7f-4804-b370-ddb28817aa8e None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:29.740924 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-cfab6996-8a7f-4804-b370-ddb28817aa8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:29.740924 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:29.740924 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:29.740924 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:29.740924 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:29.740924 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:29.740924 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:29.740924 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:29.740924 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:29.740924 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:29.740924 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:29.740924 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:29.740924 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:00:29.740924 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:00:29.740924 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:00:29.740924 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:00:29.742100 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:29.742100 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:29.742100 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:29.742100 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:29.742100 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:29.742100 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:29.742100 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:29.742100 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:29.742100 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:29.742100 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:29.742100 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:29.742100 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:29.742100 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:29.742100 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:29.742100 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:29.742100 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:29.742100 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:00:29.743267 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:00:29.743267 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:29.743267 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:29.745193 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 276/537] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:00:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:29.753630 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a34e66-8899-42db-a5f8-d778184692f6 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:29.791663 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:29.814598 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e66ee91-b652-42cc-b4e6-f3c3ba5cdc80 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:29.815026 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e66ee91-b652-42cc-b4e6-f3c3ba5cdc80 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:29.815429 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e66ee91-b652-42cc-b4e6-f3c3ba5cdc80 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:29.817274 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 277/538] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:00:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:00:29.823052 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b57a5-4f81-488b-9e98-53fb9d3777f0 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:29.823325 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b57a5-4f81-488b-9e98-53fb9d3777f0 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:29.823595 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b57a5-4f81-488b-9e98-53fb9d3777f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:29.860204 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-2e1b57a5-4f81-488b-9e98-53fb9d3777f0 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:00:29.861426 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-2e1b57a5-4f81-488b-9e98-53fb9d3777f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:00:29.890842 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a34e66-8899-42db-a5f8-d778184692f6 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:29.932464 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bbf3a9-51bf-47fa-a277-e80de8029288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:29.933081 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bbf3a9-51bf-47fa-a277-e80de8029288 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:29.938512 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bbf3a9-51bf-47fa-a277-e80de8029288 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:29.938673 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bbf3a9-51bf-47fa-a277-e80de8029288 None admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:29.939259 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bbf3a9-51bf-47fa-a277-e80de8029288 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:29.939560 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bbf3a9-51bf-47fa-a277-e80de8029288 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:29.940976 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bbf3a9-51bf-47fa-a277-e80de8029288 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:29.958315 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 262/539] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:00:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:29.963958 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bbf3a9-51bf-47fa-a277-e80de8029288 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:29.973864 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:29.992684 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 278/540] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:00:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 01:00:30.068518 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:30.103796 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bbf3a9-51bf-47fa-a277-e80de8029288 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:30.159196 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a791cd1b-8186-40f5-a023-c3d48d66ca2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:30.159835 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a791cd1b-8186-40f5-a023-c3d48d66ca2d None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:30.160113 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a791cd1b-8186-40f5-a023-c3d48d66ca2d None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:30.160405 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a791cd1b-8186-40f5-a023-c3d48d66ca2d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:30.167372 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a791cd1b-8186-40f5-a023-c3d48d66ca2d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:30.169360 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a791cd1b-8186-40f5-a023-c3d48d66ca2d None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:30.186846 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-a791cd1b-8186-40f5-a023-c3d48d66ca2d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:00:30.186846 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:30.186846 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:30.186846 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:30.186846 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:30.186846 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:30.186846 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:30.186846 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:30.186846 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:30.186846 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:30.186846 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:30.186846 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:30.186846 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 01:00:30.186846 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 01:00:30.186846 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 01:00:30.187741 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 01:00:30.187741 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:30.187741 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:30.187741 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:30.187741 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:30.187741 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:30.187741 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:30.187741 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:30.187741 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:30.187741 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:30.187741 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:30.187741 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:30.187741 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:30.187741 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:30.187741 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:30.187741 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:30.187741 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:30.188608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 01:00:30.188608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 01:00:30.188608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 01:00:30.188608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 01:00:30.188608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 01:00:30.188608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 01:00:30.188608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:00:30.188608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:30.188608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 263/541] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 01:00:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 01:00:30.193741 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1b57a5-4f81-488b-9e98-53fb9d3777f0 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:30.302564 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a791cd1b-8186-40f5-a023-c3d48d66ca2d None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:30.357636 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1b57a5-4f81-488b-9e98-53fb9d3777f0 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:30.429894 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def1ac9d-1f07-4d98-84ec-da6ef173c716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:30.431323 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def1ac9d-1f07-4d98-84ec-da6ef173c716 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:30.431791 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def1ac9d-1f07-4d98-84ec-da6ef173c716 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:30.432278 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def1ac9d-1f07-4d98-84ec-da6ef173c716 None admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:30.433181 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def1ac9d-1f07-4d98-84ec-da6ef173c716 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:30.434208 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def1ac9d-1f07-4d98-84ec-da6ef173c716 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:30.438911 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def1ac9d-1f07-4d98-84ec-da6ef173c716 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:30.461415 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 279/542] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 8 01:00:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:30.468641 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def1ac9d-1f07-4d98-84ec-da6ef173c716 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:30.478795 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a791cd1b-8186-40f5-a023-c3d48d66ca2d None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:30.531297 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ddeff0-154b-485c-ad0c-0346bf1ebbcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:30.532352 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ddeff0-154b-485c-ad0c-0346bf1ebbcd None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:30.532827 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ddeff0-154b-485c-ad0c-0346bf1ebbcd None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:30.533290 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ddeff0-154b-485c-ad0c-0346bf1ebbcd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:30.546566 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ddeff0-154b-485c-ad0c-0346bf1ebbcd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:00:30.551768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-43ddeff0-154b-485c-ad0c-0346bf1ebbcd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 01:00:30.551768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:30.551768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:30.551768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:30.551768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:30.551768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:30.551768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:30.551768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:30.551768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:30.551768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:30.551768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:30.551768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:30.551768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:00:30.551768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:00:30.551768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:00:30.551768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:00:30.552640 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:00:30.552640 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:00:30.552640 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:00:30.552640 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:00:30.552640 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:30.552640 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:30.552640 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:30.552640 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:30.552640 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:30.552640 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:30.552640 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:30.552640 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:30.552640 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:30.552640 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:30.552640 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:30.552640 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:30.553432 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:30.553432 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:30.553432 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:30.553432 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:30.553432 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:00:30.553432 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:00:30.553432 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:00:30.553432 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:00:30.553432 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:00:30.553432 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:00:30.553432 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 01:00:30.553432 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:30.560219 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:30.560689 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 264/543] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 8 01:00:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 01:00:30.562760 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ddeff0-154b-485c-ad0c-0346bf1ebbcd None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:30.596337 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:30.673554 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def1ac9d-1f07-4d98-84ec-da6ef173c716 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:30.707896 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20b65ef-481e-4413-99c7-32b4aeb67c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:30.708530 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20b65ef-481e-4413-99c7-32b4aeb67c7b None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:30.708808 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20b65ef-481e-4413-99c7-32b4aeb67c7b None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:30.709077 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20b65ef-481e-4413-99c7-32b4aeb67c7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:30.709810 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20b65ef-481e-4413-99c7-32b4aeb67c7b None admin] RBAC: Adding query filter params (role.id=609d73f56e6a48698397131168abe6df, scope.project.id=78ad2a8fb5264fb4af935e039d28c8de, user.id=5dbbcc3dbcc745458e97b31c6781ce70) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:30.710114 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20b65ef-481e-4413-99c7-32b4aeb67c7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:30.711618 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20b65ef-481e-4413-99c7-32b4aeb67c7b None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:30.717663 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 280/544] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 8 01:00:30 2022] GET /identity/v3/role_assignments?user.id=5dbbcc3dbcc745458e97b31c6781ce70&scope.project.id=78ad2a8fb5264fb4af935e039d28c8de&role.id=609d73f56e6a48698397131168abe6df => generated 259 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:30.718231 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ddeff0-154b-485c-ad0c-0346bf1ebbcd None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:30.758288 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0822446b-d27a-4cf7-b5fd-18fc11b9cb5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:30.758949 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0822446b-d27a-4cf7-b5fd-18fc11b9cb5a None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:30.759225 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0822446b-d27a-4cf7-b5fd-18fc11b9cb5a None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:30.759494 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0822446b-d27a-4cf7-b5fd-18fc11b9cb5a None admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:30.760220 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0822446b-d27a-4cf7-b5fd-18fc11b9cb5a None admin] RBAC: Adding query filter params (name=service) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:30.760523 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0822446b-d27a-4cf7-b5fd-18fc11b9cb5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:30.762740 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0822446b-d27a-4cf7-b5fd-18fc11b9cb5a None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:30.773004 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 265/545] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 8 01:00:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:30.781287 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20b65ef-481e-4413-99c7-32b4aeb67c7b None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:30.942798 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20b65ef-481e-4413-99c7-32b4aeb67c7b None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:30.971199 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:30.993798 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6966571-4156-4e8e-ab19-cd29d75020b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:30.994419 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6966571-4156-4e8e-ab19-cd29d75020b8 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:30.994694 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6966571-4156-4e8e-ab19-cd29d75020b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:30.994961 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6966571-4156-4e8e-ab19-cd29d75020b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:31.009455 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6966571-4156-4e8e-ab19-cd29d75020b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:31.011218 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6966571-4156-4e8e-ab19-cd29d75020b8 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:31.027290 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-b6966571-4156-4e8e-ab19-cd29d75020b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:31.027290 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:31.027290 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:31.027290 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:31.027290 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:31.027290 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:31.027290 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:31.027290 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:31.027290 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:31.027290 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:31.027290 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:31.027290 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:31.027290 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:00:31.027290 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:00:31.027290 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:00:31.027290 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:00:31.028852 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:31.028852 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:31.028852 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:31.028852 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:31.028852 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:31.028852 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:31.028852 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:31.028852 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:31.028852 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:31.028852 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:31.028852 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:31.028852 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:31.028852 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:31.028852 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:31.028852 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:31.028852 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:31.028852 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:00:31.029689 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:00:31.029689 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:31.029689 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:31.031593 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 281/546] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:00:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:31.034981 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0822446b-d27a-4cf7-b5fd-18fc11b9cb5a None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:31.141289 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:31.189626 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0822446b-d27a-4cf7-b5fd-18fc11b9cb5a None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:31.228048 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fd8d1e-e0fa-4e56-965d-0220763b1c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:31.228601 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fd8d1e-e0fa-4e56-965d-0220763b1c3c None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:31.228878 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fd8d1e-e0fa-4e56-965d-0220763b1c3c None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:31.229148 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fd8d1e-e0fa-4e56-965d-0220763b1c3c None admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:31.229791 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fd8d1e-e0fa-4e56-965d-0220763b1c3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:31.230089 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fd8d1e-e0fa-4e56-965d-0220763b1c3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:31.231964 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fd8d1e-e0fa-4e56-965d-0220763b1c3c None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:31.244544 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 266/547] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:00:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:31.253349 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6966571-4156-4e8e-ab19-cd29d75020b8 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:31.409255 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4286c4-25f9-403d-8f5b-afcb26f5dd6a None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:31.409769 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4286c4-25f9-403d-8f5b-afcb26f5dd6a None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:31.410283 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4286c4-25f9-403d-8f5b-afcb26f5dd6a None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:31.412881 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 267/548] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:00:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:00:31.417781 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86233f2f-9c9d-4516-889b-2ab075410892 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:31.418602 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86233f2f-9c9d-4516-889b-2ab075410892 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:31.419029 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86233f2f-9c9d-4516-889b-2ab075410892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:31.425173 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6966571-4156-4e8e-ab19-cd29d75020b8 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:31.444020 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-86233f2f-9c9d-4516-889b-2ab075410892 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:00:31.445239 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-86233f2f-9c9d-4516-889b-2ab075410892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:00:31.493877 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9949b770-a2ed-41b1-80ee-44c8e3a8f627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:31.494525 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949b770-a2ed-41b1-80ee-44c8e3a8f627 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:31.494784 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949b770-a2ed-41b1-80ee-44c8e3a8f627 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:31.495050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949b770-a2ed-41b1-80ee-44c8e3a8f627 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:31.513096 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9949b770-a2ed-41b1-80ee-44c8e3a8f627 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:31.516628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9949b770-a2ed-41b1-80ee-44c8e3a8f627 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:31.540192 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-9949b770-a2ed-41b1-80ee-44c8e3a8f627 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 08 01:00:31.540192 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:31.540192 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:31.540192 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:31.540192 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:31.540192 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:31.540192 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:31.540192 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:31.540192 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:31.540192 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:31.540192 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:31.540192 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:31.540192 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 01:00:31.540192 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 01:00:31.540192 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 01:00:31.540192 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 01:00:31.541656 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:31.541656 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:31.541656 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 01:00:31.541656 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:00:31.541656 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 01:00:31.541656 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:00:31.541656 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:31.541656 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:31.541656 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:31.541656 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:31.541656 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:31.541656 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:31.541656 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:31.541656 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:31.541656 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:31.541656 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:31.541656 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:31.541656 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:31.542717 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:31.542717 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:31.542717 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 01:00:31.542717 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 01:00:31.542717 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 01:00:31.542717 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 01:00:31.542717 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 01:00:31.542717 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 01:00:31.542717 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 08 01:00:31.542717 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:31.542717 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 282/549] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 8 01:00:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:31.547318 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9949b770-a2ed-41b1-80ee-44c8e3a8f627 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:31.593203 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 268/550] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:00:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 01:00:31.612552 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:31.694183 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9949b770-a2ed-41b1-80ee-44c8e3a8f627 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:31.740843 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab5fd9b-5725-4608-a360-8057d8536f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:31.741524 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab5fd9b-5725-4608-a360-8057d8536f34 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:31.741812 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab5fd9b-5725-4608-a360-8057d8536f34 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:31.742092 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab5fd9b-5725-4608-a360-8057d8536f34 None admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:31.742834 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab5fd9b-5725-4608-a360-8057d8536f34 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:31.743142 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab5fd9b-5725-4608-a360-8057d8536f34 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:31.744668 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab5fd9b-5725-4608-a360-8057d8536f34 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:31.769161 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 283/551] 10.222.0.42 () {56 vars in 1135 bytes} [Thu Sep 8 01:00:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:31.775535 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86233f2f-9c9d-4516-889b-2ab075410892 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:31.832435 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:31.903675 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86233f2f-9c9d-4516-889b-2ab075410892 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:31.931154 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab5fd9b-5725-4608-a360-8057d8536f34 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:31.951429 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba85ec8-87d9-4879-a948-0d735fcde369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:31.951429 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba85ec8-87d9-4879-a948-0d735fcde369 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:31.951429 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba85ec8-87d9-4879-a948-0d735fcde369 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:31.951429 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba85ec8-87d9-4879-a948-0d735fcde369 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:32.067867 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba85ec8-87d9-4879-a948-0d735fcde369 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:32.069697 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba85ec8-87d9-4879-a948-0d735fcde369 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:32.115821 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:32.132454 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab5fd9b-5725-4608-a360-8057d8536f34 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:32.163027 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:32.203870 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c726e01-b085-4e1d-a32a-12a5e4420d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:32.206160 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c726e01-b085-4e1d-a32a-12a5e4420d5d None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:32.208590 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c726e01-b085-4e1d-a32a-12a5e4420d5d None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:32.208954 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c726e01-b085-4e1d-a32a-12a5e4420d5d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:32.231501 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c726e01-b085-4e1d-a32a-12a5e4420d5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:00:32.237519 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-9c726e01-b085-4e1d-a32a-12a5e4420d5d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 01:00:32.237519 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:32.237519 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:32.237519 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:32.237519 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:32.237519 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:32.237519 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:32.237519 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:32.237519 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:32.237519 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:32.237519 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:32.237519 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:32.237519 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:00:32.237519 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:00:32.237519 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:00:32.237519 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:00:32.238534 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:00:32.238534 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:00:32.238534 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:00:32.238534 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:00:32.238534 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:32.238534 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:32.238534 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:32.238534 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:32.238534 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:32.238534 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:32.238534 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:32.238534 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:32.238534 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:32.238534 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:32.238534 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:32.238534 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:32.239919 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:32.239919 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:32.239919 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:32.239919 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:32.239919 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:00:32.239919 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:00:32.239919 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:00:32.239919 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:00:32.239919 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:00:32.239919 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:00:32.239919 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 01:00:32.239919 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:32.239919 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 284/552] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 8 01:00:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 01:00:32.245920 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c726e01-b085-4e1d-a32a-12a5e4420d5d None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:32.248972 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 269/553] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 01:00:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 01:00:32.261491 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba85ec8-87d9-4879-a948-0d735fcde369 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:32.388973 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c726e01-b085-4e1d-a32a-12a5e4420d5d None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:32.416491 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba85ec8-87d9-4879-a948-0d735fcde369 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:32.425274 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1227745-bb57-4d95-9c79-f27690d81a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:32.426150 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1227745-bb57-4d95-9c79-f27690d81a41 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:32.426626 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1227745-bb57-4d95-9c79-f27690d81a41 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:32.427118 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1227745-bb57-4d95-9c79-f27690d81a41 None admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:32.427969 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1227745-bb57-4d95-9c79-f27690d81a41 None admin] RBAC: Adding query filter params (name=service) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:32.428502 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1227745-bb57-4d95-9c79-f27690d81a41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:32.430066 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1227745-bb57-4d95-9c79-f27690d81a41 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:32.438797 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 285/554] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 8 01:00:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:32.440358 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda22c5b-334a-440f-bd45-84adbfc5903f None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:32.440869 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda22c5b-334a-440f-bd45-84adbfc5903f None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:32.441336 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda22c5b-334a-440f-bd45-84adbfc5903f None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:32.443079 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 286/555] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:00:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:00:32.445179 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda22c5b-334a-440f-bd45-84adbfc5903f None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:32.462390 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5944a559-fd6c-4fa9-a78e-ca4931e05538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:32.463404 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944a559-fd6c-4fa9-a78e-ca4931e05538 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:32.463902 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944a559-fd6c-4fa9-a78e-ca4931e05538 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:32.464410 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944a559-fd6c-4fa9-a78e-ca4931e05538 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:32.465799 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:32.476562 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5944a559-fd6c-4fa9-a78e-ca4931e05538 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:32.478263 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5944a559-fd6c-4fa9-a78e-ca4931e05538 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:32.492826 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-5944a559-fd6c-4fa9-a78e-ca4931e05538 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:32.492826 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:32.492826 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:32.492826 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:32.492826 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:32.492826 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:32.492826 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:32.492826 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:32.492826 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:32.492826 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:32.492826 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:32.492826 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:32.492826 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:00:32.492826 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:00:32.492826 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:00:32.492826 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:00:32.493826 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:32.493826 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:32.493826 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:32.493826 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:32.493826 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:32.493826 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:32.493826 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:32.493826 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:32.493826 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:32.493826 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:32.493826 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:32.493826 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:32.493826 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:32.493826 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:32.493826 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:32.493826 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:32.493826 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:00:32.494690 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:00:32.494690 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:32.494690 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:32.494690 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 270/556] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:00:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:32.495383 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b64215-636f-43f1-b338-f82e45871f81 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:32.495383 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b64215-636f-43f1-b338-f82e45871f81 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:32.495639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b64215-636f-43f1-b338-f82e45871f81 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:32.501353 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 271/557] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:00:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:00:32.501353 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bbdaea-1fb2-45d6-8cb3-abd2d8e28812 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:32.501353 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bbdaea-1fb2-45d6-8cb3-abd2d8e28812 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:32.501353 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bbdaea-1fb2-45d6-8cb3-abd2d8e28812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:32.506365 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:32.522315 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-93bbdaea-1fb2-45d6-8cb3-abd2d8e28812 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:00:32.523508 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-93bbdaea-1fb2-45d6-8cb3-abd2d8e28812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:00:32.569083 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda22c5b-334a-440f-bd45-84adbfc5903f None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:32.604163 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 272/558] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:00:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 01:00:32.604163 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bbdaea-1fb2-45d6-8cb3-abd2d8e28812 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:32.624303 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c65107-6d7d-43a2-b5a0-4668500b0926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:32.625152 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c65107-6d7d-43a2-b5a0-4668500b0926 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:32.625498 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c65107-6d7d-43a2-b5a0-4668500b0926 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:32.625838 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c65107-6d7d-43a2-b5a0-4668500b0926 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:32.642894 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c65107-6d7d-43a2-b5a0-4668500b0926 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:32.644329 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c65107-6d7d-43a2-b5a0-4668500b0926 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:32.673946 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-c3c65107-6d7d-43a2-b5a0-4668500b0926 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:32.673946 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:32.673946 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:32.673946 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:32.673946 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:32.673946 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:32.673946 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:32.673946 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:32.673946 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:32.673946 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:32.673946 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:32.673946 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:32.673946 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:00:32.673946 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:00:32.673946 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:00:32.673946 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:00:32.675196 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:32.675196 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:32.675196 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:32.675196 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:32.675196 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:32.675196 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:32.675196 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:32.675196 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:32.675196 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:32.675196 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:32.675196 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:32.675196 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:32.675196 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:32.675196 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:32.675196 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:32.675196 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:32.675196 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:00:32.676308 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:00:32.676308 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:32.676308 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:32.682986 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 287/559] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:00:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 01:00:32.682986 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f846c7c0-732d-44e7-8a92-f0ca249d3646 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:32.683226 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f846c7c0-732d-44e7-8a92-f0ca249d3646 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:32.683303 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f846c7c0-732d-44e7-8a92-f0ca249d3646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:32.703901 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-f846c7c0-732d-44e7-8a92-f0ca249d3646 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:00:32.705175 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-f846c7c0-732d-44e7-8a92-f0ca249d3646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:00:32.738646 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bbdaea-1fb2-45d6-8cb3-abd2d8e28812 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:32.779271 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:32.781501 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:32.790237 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 288/560] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:00:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 01:00:32.790627 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7067e6d3-b182-433b-83c3-850f2d214f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:32.791567 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067e6d3-b182-433b-83c3-850f2d214f5a None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:32.792042 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067e6d3-b182-433b-83c3-850f2d214f5a None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:32.792556 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067e6d3-b182-433b-83c3-850f2d214f5a None admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:32.794116 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7067e6d3-b182-433b-83c3-850f2d214f5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:32.794854 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7067e6d3-b182-433b-83c3-850f2d214f5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:32.796471 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7067e6d3-b182-433b-83c3-850f2d214f5a None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:32.797018 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f846c7c0-732d-44e7-8a92-f0ca249d3646 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:32.819600 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 273/561] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:00:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:32.824082 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7067e6d3-b182-433b-83c3-850f2d214f5a None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:32.945592 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f846c7c0-732d-44e7-8a92-f0ca249d3646 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:32.964180 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7067e6d3-b182-433b-83c3-850f2d214f5a None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:32.989722 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca30a081-2139-4041-8e75-fd423e3f35c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:32.990334 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca30a081-2139-4041-8e75-fd423e3f35c3 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:32.990618 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca30a081-2139-4041-8e75-fd423e3f35c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:32.990894 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca30a081-2139-4041-8e75-fd423e3f35c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:32.991544 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca30a081-2139-4041-8e75-fd423e3f35c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:32.991850 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca30a081-2139-4041-8e75-fd423e3f35c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:32.993270 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca30a081-2139-4041-8e75-fd423e3f35c3 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:33.006427 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 289/562] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:00:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:33.007364 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca30a081-2139-4041-8e75-fd423e3f35c3 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:33.008412 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb59c2-9a20-44ec-a267-bd9daee1f174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:33.009072 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb59c2-9a20-44ec-a267-bd9daee1f174 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:33.009351 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb59c2-9a20-44ec-a267-bd9daee1f174 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:33.009621 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb59c2-9a20-44ec-a267-bd9daee1f174 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:33.020579 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebb59c2-9a20-44ec-a267-bd9daee1f174 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:33.023142 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebb59c2-9a20-44ec-a267-bd9daee1f174 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:33.040327 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-2ebb59c2-9a20-44ec-a267-bd9daee1f174 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:00:33.040327 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:33.040327 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:33.040327 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:33.040327 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:33.040327 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:33.040327 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:33.040327 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:33.040327 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:33.040327 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:33.040327 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:33.040327 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:33.040327 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 01:00:33.040327 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 01:00:33.040327 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 01:00:33.041267 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 01:00:33.041267 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:33.041267 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:33.041267 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:33.041267 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:33.041267 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:33.041267 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:33.041267 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:33.041267 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:33.041267 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:33.041267 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:33.041267 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:33.041267 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:33.041267 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:33.041267 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:33.041267 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:33.041267 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:33.042139 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 01:00:33.042139 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 01:00:33.042139 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 01:00:33.042139 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 01:00:33.042139 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 01:00:33.042139 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 01:00:33.042139 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:00:33.042139 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:33.043778 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 274/563] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 01:00:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:33.043778 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb59c2-9a20-44ec-a267-bd9daee1f174 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:33.074129 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:33.115981 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:33.122251 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca30a081-2139-4041-8e75-fd423e3f35c3 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:33.157622 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b14099-07ce-4c65-b503-7f1fa768e39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:33.158343 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b14099-07ce-4c65-b503-7f1fa768e39e None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:33.158559 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b14099-07ce-4c65-b503-7f1fa768e39e None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:33.158775 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b14099-07ce-4c65-b503-7f1fa768e39e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:33.167467 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb59c2-9a20-44ec-a267-bd9daee1f174 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:33.169352 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b14099-07ce-4c65-b503-7f1fa768e39e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:00:33.174522 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-41b14099-07ce-4c65-b503-7f1fa768e39e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 01:00:33.174522 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:33.174522 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:33.174522 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:33.174522 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:33.174522 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:33.174522 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:33.174522 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:33.174522 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:33.174522 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:33.174522 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:33.174522 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:33.174522 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:00:33.174522 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:00:33.174522 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:00:33.174522 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:00:33.175638 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:00:33.175638 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:00:33.175638 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:00:33.175638 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:00:33.175638 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:33.175638 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:33.175638 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:33.175638 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:33.175638 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:33.175638 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:33.175638 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:33.175638 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:33.175638 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:33.175638 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:33.175638 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:33.175638 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:33.176632 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:33.176632 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:33.176632 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:33.176632 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:33.176632 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:00:33.176632 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:00:33.176632 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:00:33.176632 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:00:33.176632 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:00:33.176632 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:00:33.176632 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 01:00:33.176632 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:33.176632 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 290/564] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 8 01:00:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:33.176632 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b14099-07ce-4c65-b503-7f1fa768e39e None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:33.204313 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8434a209-8e92-4768-b5b7-d138793aec37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:33.204867 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8434a209-8e92-4768-b5b7-d138793aec37 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:33.205147 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8434a209-8e92-4768-b5b7-d138793aec37 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:33.205434 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8434a209-8e92-4768-b5b7-d138793aec37 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:33.212667 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8434a209-8e92-4768-b5b7-d138793aec37 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:33.214106 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8434a209-8e92-4768-b5b7-d138793aec37 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:33.225844 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-8434a209-8e92-4768-b5b7-d138793aec37 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 01:00:33.225844 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:33.225844 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:33.225844 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:33.225844 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:33.225844 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:33.225844 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:33.225844 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:33.225844 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:33.225844 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:33.225844 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:33.225844 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:33.225844 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 01:00:33.225844 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 01:00:33.225844 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 01:00:33.225844 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 01:00:33.226807 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:33.226807 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:33.226807 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 01:00:33.226807 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:00:33.226807 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 01:00:33.226807 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:00:33.226807 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:33.226807 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:33.226807 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:33.226807 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:33.226807 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:33.226807 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:33.226807 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:33.226807 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:33.226807 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:33.226807 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:33.226807 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:33.226807 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:33.227745 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:33.227745 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:33.227745 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 01:00:33.227745 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 01:00:33.227745 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 01:00:33.227745 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 01:00:33.227745 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 01:00:33.227745 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 01:00:33.227745 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 01:00:33.227745 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:33.227745 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 275/565] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Sep 8 01:00:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:33.227745 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8434a209-8e92-4768-b5b7-d138793aec37 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:33.280397 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b14099-07ce-4c65-b503-7f1fa768e39e None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:33.294350 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:33.314677 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d9704-4b80-4455-9b9b-8260e694bde6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:33.315273 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d9704-4b80-4455-9b9b-8260e694bde6 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:33.315546 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d9704-4b80-4455-9b9b-8260e694bde6 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:33.315811 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d9704-4b80-4455-9b9b-8260e694bde6 None admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:33.316518 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4d9704-4b80-4455-9b9b-8260e694bde6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:33.316819 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4d9704-4b80-4455-9b9b-8260e694bde6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:33.318244 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4d9704-4b80-4455-9b9b-8260e694bde6 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:33.326058 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8434a209-8e92-4768-b5b7-d138793aec37 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:33.330353 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 291/566] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 8 01:00:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:33.331501 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d309c-7679-4aa3-99b6-b94bfdc67d26 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:33.331782 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d309c-7679-4aa3-99b6-b94bfdc67d26 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:33.332051 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d309c-7679-4aa3-99b6-b94bfdc67d26 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:33.335422 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 292/567] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:00:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:00:33.336310 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d309c-7679-4aa3-99b6-b94bfdc67d26 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:33.339344 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:33.361344 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af90fe2c-af38-4c80-a6e6-d19ba6a406df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:33.361963 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af90fe2c-af38-4c80-a6e6-d19ba6a406df None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:33.362263 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af90fe2c-af38-4c80-a6e6-d19ba6a406df None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:33.362532 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af90fe2c-af38-4c80-a6e6-d19ba6a406df None admin] PATH_INFO: `/v3/roles/service` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:33.373275 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af90fe2c-af38-4c80-a6e6-d19ba6a406df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:00:33.378413 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-af90fe2c-af38-4c80-a6e6-d19ba6a406df None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 01:00:33.378413 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:33.378413 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:33.378413 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:33.378413 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:33.378413 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:33.378413 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:33.378413 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:33.378413 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:33.378413 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:33.378413 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:33.378413 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:33.378413 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:00:33.378413 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:00:33.378413 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:00:33.378413 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:00:33.379514 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:00:33.379514 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:00:33.379514 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:00:33.379514 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:00:33.379514 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:33.379514 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:33.379514 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:33.379514 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:33.379514 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:33.379514 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:33.379514 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:33.379514 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:33.379514 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:33.379514 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:33.379514 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:33.379514 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:33.380667 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:33.380667 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:33.380667 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:33.380667 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:33.380667 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:00:33.380667 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:00:33.380667 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:00:33.380667 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:00:33.380667 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:00:33.380667 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:00:33.380667 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 01:00:33.380667 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:33.380667 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 276/568] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 8 01:00:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:33.380667 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af90fe2c-af38-4c80-a6e6-d19ba6a406df None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:33.442850 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d309c-7679-4aa3-99b6-b94bfdc67d26 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:33.476841 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade2d004-4d10-47a5-8be4-795a58b4560e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:33.477364 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade2d004-4d10-47a5-8be4-795a58b4560e None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:33.477640 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade2d004-4d10-47a5-8be4-795a58b4560e None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:33.477910 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade2d004-4d10-47a5-8be4-795a58b4560e None admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:33.478565 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade2d004-4d10-47a5-8be4-795a58b4560e None admin] RBAC: Adding query filter params (name=service) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:33.478864 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade2d004-4d10-47a5-8be4-795a58b4560e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:33.478995 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af90fe2c-af38-4c80-a6e6-d19ba6a406df None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:33.480254 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade2d004-4d10-47a5-8be4-795a58b4560e None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:33.488857 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 293/569] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 8 01:00:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:33.489626 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade2d004-4d10-47a5-8be4-795a58b4560e None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:33.513745 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06c548b-50fd-4dda-b22d-4b919d3a24f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:33.514338 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c548b-50fd-4dda-b22d-4b919d3a24f5 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:33.514621 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c548b-50fd-4dda-b22d-4b919d3a24f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:33.514893 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c548b-50fd-4dda-b22d-4b919d3a24f5 None admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:33.515609 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06c548b-50fd-4dda-b22d-4b919d3a24f5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:33.515912 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06c548b-50fd-4dda-b22d-4b919d3a24f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:33.517375 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06c548b-50fd-4dda-b22d-4b919d3a24f5 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:33.525306 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:33.535070 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 277/570] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Sep 8 01:00:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:33.536289 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0f5d5-b439-4151-97e7-9975e6d71504 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:33.536575 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0f5d5-b439-4151-97e7-9975e6d71504 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:33.536850 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0f5d5-b439-4151-97e7-9975e6d71504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:33.554751 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-3bb0f5d5-b439-4151-97e7-9975e6d71504 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:00:33.555946 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-3bb0f5d5-b439-4151-97e7-9975e6d71504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:00:33.579494 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:33.592783 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade2d004-4d10-47a5-8be4-795a58b4560e None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:33.623581 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 278/571] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:00:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 01:00:33.624647 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb0f5d5-b439-4151-97e7-9975e6d71504 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:33.628170 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db96c907-8141-47a3-bf57-535978d9c851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:33.628784 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96c907-8141-47a3-bf57-535978d9c851 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:33.629067 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96c907-8141-47a3-bf57-535978d9c851 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:33.629343 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96c907-8141-47a3-bf57-535978d9c851 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:33.630076 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db96c907-8141-47a3-bf57-535978d9c851 None admin] RBAC: Adding query filter params (role.id=609d73f56e6a48698397131168abe6df, scope.project.id=78ad2a8fb5264fb4af935e039d28c8de, user.id=8add26872fe6479aa9d7de155a17195a) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:33.630390 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db96c907-8141-47a3-bf57-535978d9c851 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:33.631828 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db96c907-8141-47a3-bf57-535978d9c851 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:33.652183 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 294/572] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 8 01:00:33 2022] GET /identity/v3/role_assignments?user.id=8add26872fe6479aa9d7de155a17195a&scope.project.id=78ad2a8fb5264fb4af935e039d28c8de&role.id=609d73f56e6a48698397131168abe6df => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:33.652183 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db96c907-8141-47a3-bf57-535978d9c851 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:33.764049 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:33.766640 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db96c907-8141-47a3-bf57-535978d9c851 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:33.807812 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb0f5d5-b439-4151-97e7-9975e6d71504 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:33.822476 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54b036c-76c9-42c7-b35d-a005837257f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:33.823726 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b036c-76c9-42c7-b35d-a005837257f4 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:33.824756 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b036c-76c9-42c7-b35d-a005837257f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:33.825346 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b036c-76c9-42c7-b35d-a005837257f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:33.838736 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54b036c-76c9-42c7-b35d-a005837257f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:33.841329 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54b036c-76c9-42c7-b35d-a005837257f4 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:33.851877 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92ffbfc-99a3-42fa-ba4b-2db343af3591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:33.852564 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ffbfc-99a3-42fa-ba4b-2db343af3591 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:33.852853 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ffbfc-99a3-42fa-ba4b-2db343af3591 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:33.853132 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ffbfc-99a3-42fa-ba4b-2db343af3591 None admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:33.853800 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92ffbfc-99a3-42fa-ba4b-2db343af3591 None admin] RBAC: Adding query filter params (name=service) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:33.854109 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92ffbfc-99a3-42fa-ba4b-2db343af3591 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:33.856254 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92ffbfc-99a3-42fa-ba4b-2db343af3591 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:33.859915 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-d54b036c-76c9-42c7-b35d-a005837257f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:33.859915 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:33.859915 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:33.859915 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:33.859915 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:33.859915 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:33.859915 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:33.859915 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:33.859915 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:33.859915 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:33.859915 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:33.859915 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:33.859915 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:00:33.859915 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:00:33.859915 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:00:33.859915 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:00:33.860940 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:33.860940 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:33.860940 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:33.860940 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:33.860940 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:33.860940 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:33.860940 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:33.860940 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:33.860940 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:33.860940 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:33.860940 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:33.860940 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:33.860940 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:33.860940 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:33.860940 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:33.860940 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:33.860940 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:00:33.861826 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:00:33.861826 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:33.861826 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:33.867995 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 279/573] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 8 01:00:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:33.867995 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92ffbfc-99a3-42fa-ba4b-2db343af3591 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:33.868836 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 295/574] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:00:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:33.870799 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54b036c-76c9-42c7-b35d-a005837257f4 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:33.900722 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:33.971088 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92ffbfc-99a3-42fa-ba4b-2db343af3591 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:33.987454 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54b036c-76c9-42c7-b35d-a005837257f4 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:34.006846 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d5d946-ac13-4015-ade2-f8d14024c15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:34.007479 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d5d946-ac13-4015-ade2-f8d14024c15c None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:34.007757 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d5d946-ac13-4015-ade2-f8d14024c15c None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:34.008028 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d5d946-ac13-4015-ade2-f8d14024c15c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:34.019183 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d5d946-ac13-4015-ade2-f8d14024c15c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:34.020604 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d5d946-ac13-4015-ade2-f8d14024c15c None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:34.025077 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f828ed3-26c7-4fe9-99f3-02ba16969f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:34.025676 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f828ed3-26c7-4fe9-99f3-02ba16969f57 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:34.025950 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f828ed3-26c7-4fe9-99f3-02ba16969f57 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:34.026219 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f828ed3-26c7-4fe9-99f3-02ba16969f57 None admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:34.026851 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f828ed3-26c7-4fe9-99f3-02ba16969f57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:34.027148 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f828ed3-26c7-4fe9-99f3-02ba16969f57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:34.029479 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f828ed3-26c7-4fe9-99f3-02ba16969f57 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:34.036024 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-c4d5d946-ac13-4015-ade2-f8d14024c15c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:34.036024 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:34.036024 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:34.036024 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:34.036024 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:34.036024 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:34.036024 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:34.036024 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:34.036024 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:34.036024 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:34.036024 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:34.036024 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:34.036024 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:00:34.036024 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:00:34.036024 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:00:34.036024 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:00:34.036985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:34.036985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:34.036985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:34.036985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:34.036985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:34.036985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:34.036985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:34.036985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:34.036985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:34.036985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:34.036985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:34.036985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:34.036985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:34.036985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:34.036985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:34.036985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:34.036985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:00:34.037897 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:00:34.037897 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:34.037897 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:34.037897 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 280/575] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:00:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:34.037897 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d5d946-ac13-4015-ade2-f8d14024c15c None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:34.046536 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 296/576] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:00:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:34.047217 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f828ed3-26c7-4fe9-99f3-02ba16969f57 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:34.063592 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:34.120488 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:34.157051 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d5d946-ac13-4015-ade2-f8d14024c15c None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:34.160073 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f828ed3-26c7-4fe9-99f3-02ba16969f57 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:34.195614 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4688c5-5fa6-4950-8e03-735ddd0e0fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:34.196278 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4688c5-5fa6-4950-8e03-735ddd0e0fe3 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:34.196557 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4688c5-5fa6-4950-8e03-735ddd0e0fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:34.196658 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa1a96-9bbc-4bb7-9d1e-3181ff08246a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:34.196880 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4688c5-5fa6-4950-8e03-735ddd0e0fe3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:34.197266 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa1a96-9bbc-4bb7-9d1e-3181ff08246a None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:34.197546 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa1a96-9bbc-4bb7-9d1e-3181ff08246a None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:34.197819 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa1a96-9bbc-4bb7-9d1e-3181ff08246a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:34.208193 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fa1a96-9bbc-4bb7-9d1e-3181ff08246a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:00:34.209350 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4688c5-5fa6-4950-8e03-735ddd0e0fe3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:34.210844 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4688c5-5fa6-4950-8e03-735ddd0e0fe3 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:34.212949 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-14fa1a96-9bbc-4bb7-9d1e-3181ff08246a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 01:00:34.212949 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:34.212949 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:34.212949 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:34.212949 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:34.212949 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:34.212949 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:34.212949 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:34.212949 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:34.212949 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:34.212949 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:34.212949 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:34.212949 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:00:34.212949 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:00:34.212949 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:00:34.212949 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:00:34.213901 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:00:34.213901 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:00:34.213901 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:00:34.213901 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:00:34.213901 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:34.213901 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:34.213901 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:34.213901 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:34.213901 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:34.213901 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:34.213901 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:34.213901 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:34.213901 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:34.213901 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:34.213901 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:34.213901 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:34.214751 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:34.214751 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:34.214751 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:34.214751 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:34.214751 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:00:34.214751 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:00:34.214751 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:00:34.214751 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:00:34.214751 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:00:34.214751 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:00:34.214751 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 01:00:34.214751 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:34.214751 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 297/577] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 8 01:00:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:34.216941 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa1a96-9bbc-4bb7-9d1e-3181ff08246a None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:34.232191 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-2a4688c5-5fa6-4950-8e03-735ddd0e0fe3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:34.232191 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:34.232191 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:34.232191 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:34.232191 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:34.232191 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:34.232191 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:34.232191 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:34.232191 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:34.232191 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:34.232191 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:34.232191 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:34.232191 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:00:34.232191 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:00:34.232191 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:00:34.232191 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:00:34.233172 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:34.233172 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:34.233172 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:34.233172 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:34.233172 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:34.233172 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:34.233172 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:34.233172 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:34.233172 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:34.233172 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:34.233172 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:34.233172 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:34.233172 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:34.233172 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:34.233172 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:34.233172 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:34.233172 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:00:34.234071 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:00:34.234071 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:34.234071 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:34.234071 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 281/578] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:00:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:34.234071 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4688c5-5fa6-4950-8e03-735ddd0e0fe3 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:34.299079 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:34.331395 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa1a96-9bbc-4bb7-9d1e-3181ff08246a None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:34.358208 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4688c5-5fa6-4950-8e03-735ddd0e0fe3 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:34.368441 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ca72d8-792a-4656-9dae-24140563a77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:34.369360 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca72d8-792a-4656-9dae-24140563a77f None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:34.369696 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca72d8-792a-4656-9dae-24140563a77f None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:34.369966 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca72d8-792a-4656-9dae-24140563a77f None admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:34.370604 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ca72d8-792a-4656-9dae-24140563a77f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:34.370903 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ca72d8-792a-4656-9dae-24140563a77f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:34.372757 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ca72d8-792a-4656-9dae-24140563a77f None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:34.390232 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 298/579] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:00:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:34.390232 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ca72d8-792a-4656-9dae-24140563a77f None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:34.391256 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:34.401430 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bf8a35-9a32-46eb-8981-f4123f1ef442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:34.402059 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf8a35-9a32-46eb-8981-f4123f1ef442 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:34.402415 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf8a35-9a32-46eb-8981-f4123f1ef442 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:34.403187 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf8a35-9a32-46eb-8981-f4123f1ef442 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:34.412234 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bf8a35-9a32-46eb-8981-f4123f1ef442 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:34.413595 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bf8a35-9a32-46eb-8981-f4123f1ef442 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:34.425485 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-78bf8a35-9a32-46eb-8981-f4123f1ef442 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 08 01:00:34.425485 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:34.425485 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:34.425485 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:34.425485 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:34.425485 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:34.425485 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:34.425485 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:34.425485 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:34.425485 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:34.425485 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:34.425485 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:34.425485 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 01:00:34.425485 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 01:00:34.425485 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 01:00:34.425485 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 01:00:34.426426 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:34.426426 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:34.426426 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 01:00:34.426426 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:00:34.426426 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 01:00:34.426426 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:00:34.426426 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:34.426426 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:34.426426 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:34.426426 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:34.426426 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:34.426426 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:34.426426 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:34.426426 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:34.426426 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:34.426426 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:34.426426 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:34.426426 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:34.428049 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:34.428049 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:34.428049 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 01:00:34.428049 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 01:00:34.428049 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 01:00:34.428049 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 01:00:34.428049 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 01:00:34.428049 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 01:00:34.428049 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 08 01:00:34.428049 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:34.428049 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 282/580] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 01:00:34 2022] GET /identity/v3/users/glance => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:34.428049 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bf8a35-9a32-46eb-8981-f4123f1ef442 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:34.508167 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ca72d8-792a-4656-9dae-24140563a77f None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:34.536907 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bf8a35-9a32-46eb-8981-f4123f1ef442 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:34.545320 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:34.551409 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afd2f5c-2e10-4a05-84ca-085330349dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:34.551905 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd2f5c-2e10-4a05-84ca-085330349dcf None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:34.552210 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd2f5c-2e10-4a05-84ca-085330349dcf None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:34.552796 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd2f5c-2e10-4a05-84ca-085330349dcf None admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:34.553157 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afd2f5c-2e10-4a05-84ca-085330349dcf None admin] RBAC: Adding query filter params (name=service) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:34.553462 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afd2f5c-2e10-4a05-84ca-085330349dcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:34.554831 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afd2f5c-2e10-4a05-84ca-085330349dcf None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:34.567844 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 299/581] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 8 01:00:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:34.567844 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afd2f5c-2e10-4a05-84ca-085330349dcf None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:34.579105 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c277a15-fc13-4bc6-95dd-e22706bfa9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:34.579713 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c277a15-fc13-4bc6-95dd-e22706bfa9e4 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:34.580090 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c277a15-fc13-4bc6-95dd-e22706bfa9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:34.580291 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c277a15-fc13-4bc6-95dd-e22706bfa9e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:34.580927 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c277a15-fc13-4bc6-95dd-e22706bfa9e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:34.581236 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c277a15-fc13-4bc6-95dd-e22706bfa9e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:34.582612 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c277a15-fc13-4bc6-95dd-e22706bfa9e4 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:34.594645 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 283/582] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:00:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:34.595393 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c277a15-fc13-4bc6-95dd-e22706bfa9e4 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:34.649609 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:34.679678 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afd2f5c-2e10-4a05-84ca-085330349dcf None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:34.709208 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c277a15-fc13-4bc6-95dd-e22706bfa9e4 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:34.716716 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a197ea-335c-4052-9738-ee7f541d983e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:34.717797 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a197ea-335c-4052-9738-ee7f541d983e None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:34.718156 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a197ea-335c-4052-9738-ee7f541d983e None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:34.718506 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a197ea-335c-4052-9738-ee7f541d983e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:34.725986 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a197ea-335c-4052-9738-ee7f541d983e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:34.727509 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a197ea-335c-4052-9738-ee7f541d983e None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:34.739238 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-62a197ea-335c-4052-9738-ee7f541d983e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:00:34.739238 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:34.739238 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:34.739238 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:34.739238 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:34.739238 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:34.739238 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:34.739238 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:34.739238 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:34.739238 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:34.739238 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:34.739238 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:34.739238 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 01:00:34.739238 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 01:00:34.739238 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 01:00:34.740320 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 01:00:34.740320 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:34.740320 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:34.740320 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:34.740320 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:34.740320 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:34.740320 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:34.740320 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:34.740320 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:34.740320 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:34.740320 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:34.740320 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:34.740320 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:34.740320 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:34.740320 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:34.740320 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:34.740320 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:34.741235 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 01:00:34.741235 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 01:00:34.741235 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 01:00:34.741235 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 01:00:34.741235 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 01:00:34.741235 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 01:00:34.741235 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:00:34.741235 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:34.741235 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 300/583] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 01:00:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:34.741235 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a197ea-335c-4052-9738-ee7f541d983e None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:34.750399 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef484ace-6353-44d8-a87a-fbe788ba1349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:34.751015 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef484ace-6353-44d8-a87a-fbe788ba1349 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:34.751297 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef484ace-6353-44d8-a87a-fbe788ba1349 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:34.751567 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef484ace-6353-44d8-a87a-fbe788ba1349 None admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:34.752349 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef484ace-6353-44d8-a87a-fbe788ba1349 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:34.752659 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef484ace-6353-44d8-a87a-fbe788ba1349 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:34.754109 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef484ace-6353-44d8-a87a-fbe788ba1349 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:34.773238 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 284/584] 10.222.0.42 () {56 vars in 1137 bytes} [Thu Sep 8 01:00:34 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:34.774351 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef484ace-6353-44d8-a87a-fbe788ba1349 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:34.808152 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:34.860859 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a197ea-335c-4052-9738-ee7f541d983e None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:34.878571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef484ace-6353-44d8-a87a-fbe788ba1349 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:34.881040 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:34.900598 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21d550a-e898-4cd8-acce-6a3b039dc91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:34.901337 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d550a-e898-4cd8-acce-6a3b039dc91d None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:34.901517 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d550a-e898-4cd8-acce-6a3b039dc91d None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:34.901775 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d550a-e898-4cd8-acce-6a3b039dc91d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:34.913716 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21d550a-e898-4cd8-acce-6a3b039dc91d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:34.915193 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21d550a-e898-4cd8-acce-6a3b039dc91d None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:34.916665 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80da7ef-4541-4369-9cc4-0498b513970c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:34.917417 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80da7ef-4541-4369-9cc4-0498b513970c None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:34.918090 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80da7ef-4541-4369-9cc4-0498b513970c None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:34.918375 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80da7ef-4541-4369-9cc4-0498b513970c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:34.926568 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80da7ef-4541-4369-9cc4-0498b513970c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:34.928022 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80da7ef-4541-4369-9cc4-0498b513970c None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:34.929539 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-f21d550a-e898-4cd8-acce-6a3b039dc91d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:34.929539 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:34.929539 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:34.929539 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:34.929539 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:34.929539 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:34.929539 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:34.929539 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:34.929539 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:34.929539 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:34.929539 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:34.929539 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:34.929539 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:00:34.929539 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:00:34.929539 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:00:34.929539 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:00:34.930422 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:34.930422 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:34.930422 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:34.930422 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:34.930422 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:34.930422 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:34.930422 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:34.930422 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:34.930422 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:34.930422 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:34.930422 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:34.930422 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:34.930422 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:34.930422 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:34.930422 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:34.930422 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:34.930422 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:00:34.931329 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:00:34.931329 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:34.931329 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:34.931329 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 301/585] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:00:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:34.931560 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21d550a-e898-4cd8-acce-6a3b039dc91d None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:34.947417 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-c80da7ef-4541-4369-9cc4-0498b513970c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 01:00:34.947417 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:34.947417 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:34.947417 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:34.947417 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:34.947417 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:34.947417 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:34.947417 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:34.947417 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:34.947417 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:34.947417 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:34.947417 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:34.947417 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 01:00:34.947417 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 01:00:34.947417 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 01:00:34.947417 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 01:00:34.948474 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:34.948474 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:34.948474 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 01:00:34.948474 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:00:34.948474 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 01:00:34.948474 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:00:34.948474 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:34.948474 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:34.948474 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:34.948474 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:34.948474 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:34.948474 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:34.948474 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:34.948474 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:34.948474 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:34.948474 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:34.948474 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:34.948474 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:34.950130 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:34.950130 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:34.950130 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 01:00:34.950130 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 01:00:34.950130 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 01:00:34.950130 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 01:00:34.950130 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 01:00:34.950130 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 01:00:34.950130 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 01:00:34.950130 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:34.950130 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 285/586] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 01:00:34 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:34.952219 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80da7ef-4541-4369-9cc4-0498b513970c None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:35.048379 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21d550a-e898-4cd8-acce-6a3b039dc91d None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:35.053282 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80da7ef-4541-4369-9cc4-0498b513970c None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:35.056765 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:35.086424 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ab09c-e01c-4c30-85a9-75b92fdc5d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:35.086951 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ab09c-e01c-4c30-85a9-75b92fdc5d4b None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:35.087231 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ab09c-e01c-4c30-85a9-75b92fdc5d4b None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:35.087501 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ab09c-e01c-4c30-85a9-75b92fdc5d4b None admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:35.088217 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1ab09c-e01c-4c30-85a9-75b92fdc5d4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:35.088524 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1ab09c-e01c-4c30-85a9-75b92fdc5d4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:35.089374 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ad9287-7195-4e4d-aa7f-6c983e8b6676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:35.090510 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1ab09c-e01c-4c30-85a9-75b92fdc5d4b None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:35.090616 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad9287-7195-4e4d-aa7f-6c983e8b6676 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:35.090836 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad9287-7195-4e4d-aa7f-6c983e8b6676 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:35.091110 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad9287-7195-4e4d-aa7f-6c983e8b6676 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:35.102339 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 302/587] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 8 01:00:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:35.103130 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ab09c-e01c-4c30-85a9-75b92fdc5d4b None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:35.109972 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ad9287-7195-4e4d-aa7f-6c983e8b6676 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:35.111449 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ad9287-7195-4e4d-aa7f-6c983e8b6676 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:35.127114 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-44ad9287-7195-4e4d-aa7f-6c983e8b6676 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:35.127114 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:35.127114 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:35.127114 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:35.127114 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:35.127114 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:35.127114 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:35.127114 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:35.127114 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:35.127114 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:35.127114 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:35.127114 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:35.127114 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:00:35.127114 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:00:35.127114 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:00:35.127114 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:00:35.128049 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:35.128049 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:35.128049 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:35.128049 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:35.128049 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:35.128049 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:35.128049 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:35.128049 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:35.128049 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:35.128049 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:35.128049 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:35.128049 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:35.128049 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:35.128049 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:35.128049 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:35.128049 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:35.128049 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:00:35.128970 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:00:35.128970 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:35.128970 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:35.128970 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 286/588] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:00:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:35.128970 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ad9287-7195-4e4d-aa7f-6c983e8b6676 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:35.148874 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:35.220579 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ab09c-e01c-4c30-85a9-75b92fdc5d4b None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:35.233899 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ad9287-7195-4e4d-aa7f-6c983e8b6676 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:35.258103 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fde75a-df47-42f6-8453-d1a2204d123e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:35.258770 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fde75a-df47-42f6-8453-d1a2204d123e None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:35.259060 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fde75a-df47-42f6-8453-d1a2204d123e None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:35.259328 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fde75a-df47-42f6-8453-d1a2204d123e None admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:35.259993 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fde75a-df47-42f6-8453-d1a2204d123e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:35.260328 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fde75a-df47-42f6-8453-d1a2204d123e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:35.262483 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fde75a-df47-42f6-8453-d1a2204d123e None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:35.270383 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc97fd93-098c-4173-8918-977f1aebac3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:35.271105 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97fd93-098c-4173-8918-977f1aebac3a None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:35.271317 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97fd93-098c-4173-8918-977f1aebac3a None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:35.271554 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97fd93-098c-4173-8918-977f1aebac3a None admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:35.272309 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc97fd93-098c-4173-8918-977f1aebac3a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:35.272639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc97fd93-098c-4173-8918-977f1aebac3a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:35.274046 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc97fd93-098c-4173-8918-977f1aebac3a None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:35.274818 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 303/589] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:00:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:35.275535 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fde75a-df47-42f6-8453-d1a2204d123e None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:35.297219 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 287/590] 10.222.0.42 () {56 vars in 1137 bytes} [Thu Sep 8 01:00:35 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:35.301384 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc97fd93-098c-4173-8918-977f1aebac3a None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:35.304943 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:35.382848 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:35.383885 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fde75a-df47-42f6-8453-d1a2204d123e None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:35.410589 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc97fd93-098c-4173-8918-977f1aebac3a None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:35.421239 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983530f7-b8d1-454c-be08-40ab4b178e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:35.421903 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983530f7-b8d1-454c-be08-40ab4b178e42 None admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:35.422184 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983530f7-b8d1-454c-be08-40ab4b178e42 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:35.422461 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983530f7-b8d1-454c-be08-40ab4b178e42 None admin] PATH_INFO: `/v3/projects/78ad2a8fb5264fb4af935e039d28c8de/users/ae643cfdcb164aa9b474747cd21aa116/roles/609d73f56e6a48698397131168abe6df` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:35.439454 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983530f7-b8d1-454c-be08-40ab4b178e42 None admin] RBAC: Authorizing `identity:create_grant(project_id=78ad2a8fb5264fb4af935e039d28c8de, user_id=ae643cfdcb164aa9b474747cd21aa116, role_id=609d73f56e6a48698397131168abe6df)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:35.441048 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983530f7-b8d1-454c-be08-40ab4b178e42 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:35.451359 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7691faa2-e3e1-4007-bc7e-249af0ec551b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:35.452210 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7691faa2-e3e1-4007-bc7e-249af0ec551b None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:35.452667 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7691faa2-e3e1-4007-bc7e-249af0ec551b None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:35.452940 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7691faa2-e3e1-4007-bc7e-249af0ec551b None admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:35.453607 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7691faa2-e3e1-4007-bc7e-249af0ec551b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:35.453922 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7691faa2-e3e1-4007-bc7e-249af0ec551b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:35.464472 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 304/591] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 8 01:00:35 2022] PUT /identity/v3/projects/78ad2a8fb5264fb4af935e039d28c8de/users/ae643cfdcb164aa9b474747cd21aa116/roles/609d73f56e6a48698397131168abe6df => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:00:35.464472 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983530f7-b8d1-454c-be08-40ab4b178e42 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:35.465276 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7691faa2-e3e1-4007-bc7e-249af0ec551b None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:35.469116 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 288/592] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:00:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:35.470062 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7691faa2-e3e1-4007-bc7e-249af0ec551b None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:35.584317 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:35.607211 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7691faa2-e3e1-4007-bc7e-249af0ec551b None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:35.613321 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983530f7-b8d1-454c-be08-40ab4b178e42 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:35.656594 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce068a9-d840-4a37-ad64-d18d64b71f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:35.657380 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce068a9-d840-4a37-ad64-d18d64b71f1e None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:35.657753 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce068a9-d840-4a37-ad64-d18d64b71f1e None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:35.658130 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce068a9-d840-4a37-ad64-d18d64b71f1e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:35.659261 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dec999-7007-4bca-aab6-aedbcaea1229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:35.659896 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dec999-7007-4bca-aab6-aedbcaea1229 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:35.660341 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dec999-7007-4bca-aab6-aedbcaea1229 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:35.660617 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dec999-7007-4bca-aab6-aedbcaea1229 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:35.662691 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:35.666644 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce068a9-d840-4a37-ad64-d18d64b71f1e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:35.668388 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce068a9-d840-4a37-ad64-d18d64b71f1e None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:35.675035 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dec999-7007-4bca-aab6-aedbcaea1229 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:35.676815 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dec999-7007-4bca-aab6-aedbcaea1229 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:35.683267 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-bce068a9-d840-4a37-ad64-d18d64b71f1e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 08 01:00:35.683267 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:35.683267 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:35.683267 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:35.683267 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:35.683267 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:35.683267 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:35.683267 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:35.683267 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:35.683267 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:35.683267 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:35.683267 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:35.683267 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 01:00:35.683267 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 01:00:35.683267 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 01:00:35.683267 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 01:00:35.685231 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:35.685231 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:35.685231 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 01:00:35.685231 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:00:35.685231 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 01:00:35.685231 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:00:35.685231 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:35.685231 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:35.685231 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:35.685231 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:35.685231 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:35.685231 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:35.685231 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:35.685231 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:35.685231 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:35.685231 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:35.685231 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:35.685231 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:35.686210 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:35.686210 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:35.686210 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 01:00:35.686210 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 01:00:35.686210 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 01:00:35.686210 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 01:00:35.686210 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 01:00:35.686210 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 01:00:35.686210 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 08 01:00:35.686210 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:35.686210 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 305/593] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 8 01:00:35 2022] GET /identity/v3/users/neutron => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:35.687036 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce068a9-d840-4a37-ad64-d18d64b71f1e None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:35.695319 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-d4dec999-7007-4bca-aab6-aedbcaea1229 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:35.695319 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:35.695319 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:35.695319 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:35.695319 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:35.695319 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:35.695319 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:35.695319 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:35.695319 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:35.695319 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:35.695319 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:35.695319 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:35.695319 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:00:35.695319 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:00:35.695319 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:00:35.695319 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:00:35.697550 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:35.697550 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:35.697550 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:35.697550 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:35.697550 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:35.697550 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:35.697550 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:35.697550 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:35.697550 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:35.697550 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:35.697550 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:35.697550 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:35.697550 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:35.697550 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:35.697550 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:35.697550 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:35.697550 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:00:35.699243 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:00:35.699243 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:35.699243 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:35.699243 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 289/594] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:00:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:35.699524 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dec999-7007-4bca-aab6-aedbcaea1229 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:35.821756 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dec999-7007-4bca-aab6-aedbcaea1229 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:35.822947 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce068a9-d840-4a37-ad64-d18d64b71f1e None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:35.858722 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75db62a-b132-454f-914c-4bca76b6e645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:35.859603 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75db62a-b132-454f-914c-4bca76b6e645 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:35.859886 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75db62a-b132-454f-914c-4bca76b6e645 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:35.860172 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75db62a-b132-454f-914c-4bca76b6e645 None admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:35.860903 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75db62a-b132-454f-914c-4bca76b6e645 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:35.861006 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd40a986-f033-456c-b314-5f5bb6e00803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:35.861211 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75db62a-b132-454f-914c-4bca76b6e645 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:35.861641 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd40a986-f033-456c-b314-5f5bb6e00803 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:35.861965 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd40a986-f033-456c-b314-5f5bb6e00803 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:35.862290 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd40a986-f033-456c-b314-5f5bb6e00803 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:35.862640 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75db62a-b132-454f-914c-4bca76b6e645 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:35.865278 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:35.870869 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd40a986-f033-456c-b314-5f5bb6e00803 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:35.872668 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd40a986-f033-456c-b314-5f5bb6e00803 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:35.887792 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 290/595] 10.222.0.42 () {56 vars in 1139 bytes} [Thu Sep 8 01:00:35 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:35.887792 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75db62a-b132-454f-914c-4bca76b6e645 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:35.888361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-cd40a986-f033-456c-b314-5f5bb6e00803 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:00:35.888361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:35.888361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:35.888361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:35.888361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:35.888361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:35.888361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:35.888361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:35.888361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:35.888361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:35.888361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:35.888361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:35.888361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 01:00:35.888361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 01:00:35.888361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 01:00:35.889201 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 01:00:35.889201 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:35.889201 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:35.889201 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:35.889201 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:35.889201 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:35.889201 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:35.889201 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:35.889201 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:35.889201 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:35.889201 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:35.889201 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:35.889201 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:35.889201 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:35.889201 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:35.889201 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:35.889201 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:35.890104 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 01:00:35.890104 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 01:00:35.890104 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 01:00:35.890104 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 01:00:35.890104 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 01:00:35.890104 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 01:00:35.890104 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:00:35.890104 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:35.890104 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 306/596] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 01:00:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:35.890104 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d6b01-9bd7-4e62-86eb-64df51defdcb None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:35.890104 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d6b01-9bd7-4e62-86eb-64df51defdcb None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:35.890693 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d6b01-9bd7-4e62-86eb-64df51defdcb None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:35.891372 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 307/597] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:00:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:00:35.892041 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d6b01-9bd7-4e62-86eb-64df51defdcb None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:35.938964 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:35.999938 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75db62a-b132-454f-914c-4bca76b6e645 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:36.010129 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d6b01-9bd7-4e62-86eb-64df51defdcb None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:36.044031 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bc8cdf-7e16-4b30-b7e4-87a4456094fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:36.044669 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc8cdf-7e16-4b30-b7e4-87a4456094fc None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:36.044958 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc8cdf-7e16-4b30-b7e4-87a4456094fc None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:36.045215 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc8cdf-7e16-4b30-b7e4-87a4456094fc None admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:36.045853 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bc8cdf-7e16-4b30-b7e4-87a4456094fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:36.046154 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bc8cdf-7e16-4b30-b7e4-87a4456094fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:36.046692 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a353b1-b927-4a38-90cd-e90501e987a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:36.047315 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a353b1-b927-4a38-90cd-e90501e987a5 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:36.047532 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bc8cdf-7e16-4b30-b7e4-87a4456094fc None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:36.047637 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a353b1-b927-4a38-90cd-e90501e987a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:36.047845 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a353b1-b927-4a38-90cd-e90501e987a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:36.058658 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a353b1-b927-4a38-90cd-e90501e987a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:36.063930 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 291/598] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:00:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:36.063930 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bc8cdf-7e16-4b30-b7e4-87a4456094fc None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:36.064161 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a353b1-b927-4a38-90cd-e90501e987a5 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:36.081451 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-66a353b1-b927-4a38-90cd-e90501e987a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:36.081451 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:36.081451 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:36.081451 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:36.081451 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:36.081451 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:36.081451 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:36.081451 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:36.081451 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:36.081451 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:36.081451 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:36.081451 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:36.081451 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:00:36.081451 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:00:36.081451 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:00:36.081451 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:00:36.082715 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:36.082715 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:36.082715 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:36.082715 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:36.082715 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:36.082715 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:36.082715 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:36.082715 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:36.082715 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:36.082715 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:36.082715 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:36.082715 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:36.082715 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:36.082715 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:36.082715 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:36.082715 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:36.082715 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:00:36.088448 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:00:36.088448 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:36.088448 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:36.088448 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 308/599] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:00:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:36.088448 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcea480-fb9d-4367-a621-837f6864c22b None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:36.088448 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcea480-fb9d-4367-a621-837f6864c22b None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:36.088448 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcea480-fb9d-4367-a621-837f6864c22b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:36.104565 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-5fcea480-fb9d-4367-a621-837f6864c22b None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:00:36.106041 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-5fcea480-fb9d-4367-a621-837f6864c22b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:00:36.154986 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:36.177884 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bc8cdf-7e16-4b30-b7e4-87a4456094fc None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:36.184171 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 309/600] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:00:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 01:00:36.184171 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcea480-fb9d-4367-a621-837f6864c22b None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:36.184858 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:36.229592 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074aef80-0acd-438e-b737-191953279381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:36.230302 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074aef80-0acd-438e-b737-191953279381 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:36.230673 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074aef80-0acd-438e-b737-191953279381 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:36.231050 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074aef80-0acd-438e-b737-191953279381 None admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:36.231818 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074aef80-0acd-438e-b737-191953279381 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:36.233420 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074aef80-0acd-438e-b737-191953279381 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:36.236488 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074aef80-0acd-438e-b737-191953279381 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:36.254648 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 292/601] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 8 01:00:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:36.260420 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074aef80-0acd-438e-b737-191953279381 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:36.312983 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcea480-fb9d-4367-a621-837f6864c22b None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:36.350651 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef193a13-5323-431d-892c-fc0d7f9a37fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:36.351433 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef193a13-5323-431d-892c-fc0d7f9a37fa None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:36.351812 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef193a13-5323-431d-892c-fc0d7f9a37fa None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:36.352214 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef193a13-5323-431d-892c-fc0d7f9a37fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:36.359817 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef193a13-5323-431d-892c-fc0d7f9a37fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:36.361515 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef193a13-5323-431d-892c-fc0d7f9a37fa None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:36.369895 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074aef80-0acd-438e-b737-191953279381 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:36.374607 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-ef193a13-5323-431d-892c-fc0d7f9a37fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:00:36.374607 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:36.374607 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:36.374607 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:36.374607 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:36.374607 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:36.374607 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:36.374607 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:36.374607 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:36.374607 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:36.374607 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:36.374607 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:36.374607 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 01:00:36.374607 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 01:00:36.374607 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 01:00:36.375664 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 01:00:36.375664 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:36.375664 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:36.375664 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:36.375664 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:36.375664 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:36.375664 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:36.375664 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:36.375664 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:36.375664 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:36.375664 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:36.375664 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:36.375664 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:36.375664 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:36.375664 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:36.375664 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:36.375664 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:36.376819 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 01:00:36.376819 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 01:00:36.376819 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 01:00:36.376819 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 01:00:36.376819 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 01:00:36.376819 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 01:00:36.376819 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:00:36.376819 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:36.378333 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 310/602] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 01:00:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:36.383625 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef193a13-5323-431d-892c-fc0d7f9a37fa None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:36.411514 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2038b6b9-0b6f-4a77-b854-424fa4e41163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:36.412506 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2038b6b9-0b6f-4a77-b854-424fa4e41163 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:36.413040 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2038b6b9-0b6f-4a77-b854-424fa4e41163 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:36.413475 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2038b6b9-0b6f-4a77-b854-424fa4e41163 None admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:36.414313 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2038b6b9-0b6f-4a77-b854-424fa4e41163 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:36.414774 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2038b6b9-0b6f-4a77-b854-424fa4e41163 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:36.416518 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2038b6b9-0b6f-4a77-b854-424fa4e41163 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:36.430724 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 293/603] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:00:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:36.431751 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2038b6b9-0b6f-4a77-b854-424fa4e41163 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:36.431869 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:36.466246 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:36.505937 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef193a13-5323-431d-892c-fc0d7f9a37fa None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:36.550061 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2038b6b9-0b6f-4a77-b854-424fa4e41163 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:36.550454 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55ab811-6906-49c7-86b4-f4c066b932fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:36.551185 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ab811-6906-49c7-86b4-f4c066b932fc None admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:36.551786 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ab811-6906-49c7-86b4-f4c066b932fc None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:36.552159 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ab811-6906-49c7-86b4-f4c066b932fc None admin] PATH_INFO: `/v3/projects/78ad2a8fb5264fb4af935e039d28c8de/users/d1acfa7a09d34e2a9045c3f3fbea0f52/roles/609d73f56e6a48698397131168abe6df` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:36.570871 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55ab811-6906-49c7-86b4-f4c066b932fc None admin] RBAC: Authorizing `identity:create_grant(project_id=78ad2a8fb5264fb4af935e039d28c8de, user_id=d1acfa7a09d34e2a9045c3f3fbea0f52, role_id=609d73f56e6a48698397131168abe6df)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:36.572769 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55ab811-6906-49c7-86b4-f4c066b932fc None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:36.587913 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 311/604] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 8 01:00:36 2022] PUT /identity/v3/projects/78ad2a8fb5264fb4af935e039d28c8de/users/d1acfa7a09d34e2a9045c3f3fbea0f52/roles/609d73f56e6a48698397131168abe6df => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:00:36.588746 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55ab811-6906-49c7-86b4-f4c066b932fc None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:36.600876 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea8d05-efd7-4d26-96d3-743435706269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:36.601512 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea8d05-efd7-4d26-96d3-743435706269 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:36.604530 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea8d05-efd7-4d26-96d3-743435706269 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:36.605100 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea8d05-efd7-4d26-96d3-743435706269 None admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:36.606912 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ea8d05-efd7-4d26-96d3-743435706269 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:36.607491 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ea8d05-efd7-4d26-96d3-743435706269 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:36.609808 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ea8d05-efd7-4d26-96d3-743435706269 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:36.630067 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 294/605] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 8 01:00:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:36.631373 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea8d05-efd7-4d26-96d3-743435706269 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:36.722597 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:36.738983 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55ab811-6906-49c7-86b4-f4c066b932fc None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:36.764533 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:36.780599 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea8d05-efd7-4d26-96d3-743435706269 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:36.782804 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b9d2e-ffa3-4dcb-9e03-c964eb4f51e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:36.783574 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b9d2e-ffa3-4dcb-9e03-c964eb4f51e2 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:36.783916 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b9d2e-ffa3-4dcb-9e03-c964eb4f51e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:36.784302 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b9d2e-ffa3-4dcb-9e03-c964eb4f51e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:36.791954 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6b9d2e-ffa3-4dcb-9e03-c964eb4f51e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:36.794729 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6b9d2e-ffa3-4dcb-9e03-c964eb4f51e2 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:36.807540 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-3c6b9d2e-ffa3-4dcb-9e03-c964eb4f51e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:00:36.807540 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:36.807540 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:36.807540 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:36.807540 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:36.807540 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:36.807540 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:36.807540 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:36.807540 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:36.807540 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:36.807540 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:36.807540 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:36.807540 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 01:00:36.807540 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 01:00:36.807540 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 01:00:36.808825 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 01:00:36.808825 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:36.808825 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:36.808825 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:36.808825 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:36.808825 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:36.808825 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:36.808825 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:36.808825 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:36.808825 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:36.808825 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:36.808825 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:36.808825 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:36.808825 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:36.808825 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:36.808825 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:36.808825 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:36.809866 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 01:00:36.809866 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 01:00:36.809866 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 01:00:36.809866 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 01:00:36.809866 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 01:00:36.809866 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 01:00:36.809866 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:00:36.809866 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:36.809866 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 312/606] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 01:00:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:36.810787 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b9d2e-ffa3-4dcb-9e03-c964eb4f51e2 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:36.830823 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fd0edc-6e1f-45f5-aed0-72a641d6a0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:36.831741 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fd0edc-6e1f-45f5-aed0-72a641d6a0d8 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:36.832009 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fd0edc-6e1f-45f5-aed0-72a641d6a0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:36.832316 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fd0edc-6e1f-45f5-aed0-72a641d6a0d8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:36.847823 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fd0edc-6e1f-45f5-aed0-72a641d6a0d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:00:36.852773 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-80fd0edc-6e1f-45f5-aed0-72a641d6a0d8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 01:00:36.852773 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:36.852773 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:36.852773 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:36.852773 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:36.852773 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:36.852773 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:36.852773 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:36.852773 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:36.852773 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:36.852773 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:36.852773 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:36.852773 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:00:36.852773 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:00:36.852773 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:00:36.852773 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:00:36.853720 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:00:36.853720 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:00:36.853720 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:00:36.853720 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:00:36.853720 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:36.853720 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:36.853720 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:36.853720 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:36.853720 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:36.853720 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:36.853720 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:36.853720 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:36.853720 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:36.853720 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:36.853720 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:36.853720 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:36.854582 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:36.854582 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:36.854582 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:36.854582 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:36.854582 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:00:36.854582 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:00:36.854582 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:00:36.854582 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:00:36.854582 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:00:36.854582 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:00:36.854582 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 01:00:36.854582 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:36.854582 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 295/607] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 8 01:00:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:36.855307 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fd0edc-6e1f-45f5-aed0-72a641d6a0d8 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:36.957815 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b9d2e-ffa3-4dcb-9e03-c964eb4f51e2 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:36.967565 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fd0edc-6e1f-45f5-aed0-72a641d6a0d8 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:37.001065 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d119c7c-e315-4135-b912-534f79163c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:37.001729 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d119c7c-e315-4135-b912-534f79163c04 None admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:37.002004 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d119c7c-e315-4135-b912-534f79163c04 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:37.002276 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d119c7c-e315-4135-b912-534f79163c04 None admin] PATH_INFO: `/v3/projects/78ad2a8fb5264fb4af935e039d28c8de/users/a307e9150ead4877b970e09f36553968/roles/609d73f56e6a48698397131168abe6df` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:37.010476 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fee8f1-12bd-4b6f-a700-c35be6263bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:37.012228 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fee8f1-12bd-4b6f-a700-c35be6263bf9 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:37.012709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fee8f1-12bd-4b6f-a700-c35be6263bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:37.013181 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fee8f1-12bd-4b6f-a700-c35be6263bf9 None admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:37.014345 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fee8f1-12bd-4b6f-a700-c35be6263bf9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:37.015101 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fee8f1-12bd-4b6f-a700-c35be6263bf9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:37.017599 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fee8f1-12bd-4b6f-a700-c35be6263bf9 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:37.018903 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d119c7c-e315-4135-b912-534f79163c04 None admin] RBAC: Authorizing `identity:create_grant(project_id=78ad2a8fb5264fb4af935e039d28c8de, user_id=a307e9150ead4877b970e09f36553968, role_id=609d73f56e6a48698397131168abe6df)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:37.020876 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d119c7c-e315-4135-b912-534f79163c04 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:37.032513 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:37.033860 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 296/608] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 8 01:00:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:37.034913 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fee8f1-12bd-4b6f-a700-c35be6263bf9 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:37.040003 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:37.045917 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 313/609] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 8 01:00:36 2022] PUT /identity/v3/projects/78ad2a8fb5264fb4af935e039d28c8de/users/a307e9150ead4877b970e09f36553968/roles/609d73f56e6a48698397131168abe6df => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:00:37.052705 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d119c7c-e315-4135-b912-534f79163c04 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:37.164295 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fee8f1-12bd-4b6f-a700-c35be6263bf9 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:37.204407 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bbbd6e-a501-43cc-bbac-5a5e4ce3c5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:37.205139 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bbbd6e-a501-43cc-bbac-5a5e4ce3c5cd None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:37.205493 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bbbd6e-a501-43cc-bbac-5a5e4ce3c5cd None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:37.205839 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bbbd6e-a501-43cc-bbac-5a5e4ce3c5cd None admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:37.206584 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bbbd6e-a501-43cc-bbac-5a5e4ce3c5cd None admin] RBAC: Adding query filter params (name=service) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:37.206955 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bbbd6e-a501-43cc-bbac-5a5e4ce3c5cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:37.208453 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bbbd6e-a501-43cc-bbac-5a5e4ce3c5cd None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:37.218959 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 297/610] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 8 01:00:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:37.224532 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bbbd6e-a501-43cc-bbac-5a5e4ce3c5cd None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:37.263826 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d119c7c-e315-4135-b912-534f79163c04 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:37.308701 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75c0d63-d484-4ecb-8971-2f4920791632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:37.309444 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c0d63-d484-4ecb-8971-2f4920791632 None admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:37.309742 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c0d63-d484-4ecb-8971-2f4920791632 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:37.309983 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c0d63-d484-4ecb-8971-2f4920791632 None admin] PATH_INFO: `/v3/projects/78ad2a8fb5264fb4af935e039d28c8de/users/5dbbcc3dbcc745458e97b31c6781ce70/roles/609d73f56e6a48698397131168abe6df` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:37.316473 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:37.326692 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75c0d63-d484-4ecb-8971-2f4920791632 None admin] RBAC: Authorizing `identity:create_grant(project_id=78ad2a8fb5264fb4af935e039d28c8de, user_id=5dbbcc3dbcc745458e97b31c6781ce70, role_id=609d73f56e6a48698397131168abe6df)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:37.328187 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75c0d63-d484-4ecb-8971-2f4920791632 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:37.346230 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 314/611] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 8 01:00:37 2022] PUT /identity/v3/projects/78ad2a8fb5264fb4af935e039d28c8de/users/5dbbcc3dbcc745458e97b31c6781ce70/roles/609d73f56e6a48698397131168abe6df => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:00:37.347786 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bbbd6e-a501-43cc-bbac-5a5e4ce3c5cd None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:37.390029 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33468fa-9240-4bf1-919b-52f4069f5853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:37.390694 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33468fa-9240-4bf1-919b-52f4069f5853 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:37.390981 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33468fa-9240-4bf1-919b-52f4069f5853 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:37.391255 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33468fa-9240-4bf1-919b-52f4069f5853 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:37.404054 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33468fa-9240-4bf1-919b-52f4069f5853 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:37.405706 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33468fa-9240-4bf1-919b-52f4069f5853 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:37.422815 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-b33468fa-9240-4bf1-919b-52f4069f5853 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:37.422815 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:37.422815 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:37.422815 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:37.422815 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:37.422815 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:37.422815 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:37.422815 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:37.422815 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:37.422815 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:37.422815 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:37.422815 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:37.422815 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:00:37.422815 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:00:37.422815 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:00:37.422815 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:00:37.423808 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:37.423808 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:37.423808 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:37.423808 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:37.423808 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:37.423808 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:37.423808 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:37.423808 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:37.423808 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:37.423808 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:37.423808 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:37.423808 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:37.423808 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:37.423808 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:37.423808 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:37.423808 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:37.423808 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:00:37.424720 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:00:37.424720 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:37.424720 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:37.424720 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 298/612] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:00:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:37.429734 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75c0d63-d484-4ecb-8971-2f4920791632 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:37.514761 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:37.580240 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75c0d63-d484-4ecb-8971-2f4920791632 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:37.619026 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc216db6-3334-4bc9-bfc3-8b0225ab399f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:37.619642 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc216db6-3334-4bc9-bfc3-8b0225ab399f None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:37.619921 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc216db6-3334-4bc9-bfc3-8b0225ab399f None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:37.620211 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc216db6-3334-4bc9-bfc3-8b0225ab399f None admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:37.620867 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc216db6-3334-4bc9-bfc3-8b0225ab399f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:37.621385 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc216db6-3334-4bc9-bfc3-8b0225ab399f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:37.622803 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc216db6-3334-4bc9-bfc3-8b0225ab399f None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:37.635309 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 315/613] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:00:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:37.641074 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33468fa-9240-4bf1-919b-52f4069f5853 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:37.785193 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33468fa-9240-4bf1-919b-52f4069f5853 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:37.816484 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:37.835724 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97274c77-178d-48b0-83c9-3314a14ba43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:37.836405 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97274c77-178d-48b0-83c9-3314a14ba43b None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:37.836686 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97274c77-178d-48b0-83c9-3314a14ba43b None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:37.836958 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97274c77-178d-48b0-83c9-3314a14ba43b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:37.845392 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97274c77-178d-48b0-83c9-3314a14ba43b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:37.846859 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97274c77-178d-48b0-83c9-3314a14ba43b None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:37.865384 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-97274c77-178d-48b0-83c9-3314a14ba43b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 08 01:00:37.865384 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:37.865384 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:37.865384 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:37.865384 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:37.865384 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:37.865384 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:37.865384 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:37.865384 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:37.865384 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:37.865384 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:37.865384 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:37.865384 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 01:00:37.865384 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 01:00:37.865384 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 01:00:37.865384 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 01:00:37.866585 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:37.866585 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:37.866585 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 01:00:37.866585 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:00:37.866585 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 01:00:37.866585 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:00:37.866585 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:37.866585 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:37.866585 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:37.866585 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:37.866585 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:37.866585 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:37.866585 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:37.866585 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:37.866585 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:37.866585 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:37.866585 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:37.866585 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:37.867755 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:37.867755 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:37.867755 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 01:00:37.867755 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 01:00:37.867755 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 01:00:37.867755 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 01:00:37.867755 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 01:00:37.867755 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 01:00:37.867755 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 08 01:00:37.867755 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:37.867755 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 299/614] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 8 01:00:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:37.874255 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc216db6-3334-4bc9-bfc3-8b0225ab399f None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:37.992115 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc216db6-3334-4bc9-bfc3-8b0225ab399f None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:38.013168 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:38.027821 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994be08a-1782-4949-8828-2669d0b16b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:38.028612 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994be08a-1782-4949-8828-2669d0b16b88 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:38.028960 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994be08a-1782-4949-8828-2669d0b16b88 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:38.029510 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994be08a-1782-4949-8828-2669d0b16b88 None admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:38.030248 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994be08a-1782-4949-8828-2669d0b16b88 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:38.030552 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994be08a-1782-4949-8828-2669d0b16b88 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:38.031981 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994be08a-1782-4949-8828-2669d0b16b88 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:38.051029 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 316/615] 10.222.0.42 () {56 vars in 1135 bytes} [Thu Sep 8 01:00:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:38.056284 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97274c77-178d-48b0-83c9-3314a14ba43b None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:38.067979 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88006211-2f2b-427c-8e71-d8d980189a33 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:38.068377 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88006211-2f2b-427c-8e71-d8d980189a33 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:38.068723 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88006211-2f2b-427c-8e71-d8d980189a33 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:38.070393 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 317/616] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:00:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 01:00:38.076319 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da06ab0-6cfb-4ebc-ab23-372481fa5d3a None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:38.076688 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da06ab0-6cfb-4ebc-ab23-372481fa5d3a None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:38.077031 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da06ab0-6cfb-4ebc-ab23-372481fa5d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:38.095988 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-7da06ab0-6cfb-4ebc-ab23-372481fa5d3a None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:00:38.097315 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-7da06ab0-6cfb-4ebc-ab23-372481fa5d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:00:38.208249 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 318/617] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:00:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 01:00:38.239888 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97274c77-178d-48b0-83c9-3314a14ba43b None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:38.275763 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d0dbb6-b0c4-4bfe-80e8-68b640df2c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:38.276523 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d0dbb6-b0c4-4bfe-80e8-68b640df2c0e None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:38.276873 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d0dbb6-b0c4-4bfe-80e8-68b640df2c0e None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:38.277249 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d0dbb6-b0c4-4bfe-80e8-68b640df2c0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:38.288435 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d0dbb6-b0c4-4bfe-80e8-68b640df2c0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:38.289867 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d0dbb6-b0c4-4bfe-80e8-68b640df2c0e None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:38.304950 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-e9d0dbb6-b0c4-4bfe-80e8-68b640df2c0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:38.304950 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:38.304950 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:38.304950 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:38.304950 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:38.304950 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:38.304950 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:38.304950 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:38.304950 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:38.304950 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:38.304950 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:38.304950 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:38.304950 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:00:38.304950 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:00:38.304950 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:00:38.304950 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:00:38.306913 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:38.306913 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:38.306913 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:38.306913 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:38.306913 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:38.306913 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:38.306913 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:38.306913 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:38.306913 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:38.306913 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:38.306913 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:38.306913 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:38.306913 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:38.306913 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:38.306913 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:38.306913 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:38.306913 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:00:38.308590 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:00:38.308590 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:38.308590 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:38.308590 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 300/618] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:00:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:38.313469 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da06ab0-6cfb-4ebc-ab23-372481fa5d3a None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:38.413231 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:38.420029 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da06ab0-6cfb-4ebc-ab23-372481fa5d3a None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:38.455637 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f891bcde-3e1f-4ae7-b589-8d379dda7d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:38.456370 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f891bcde-3e1f-4ae7-b589-8d379dda7d68 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:38.456720 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f891bcde-3e1f-4ae7-b589-8d379dda7d68 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:38.457110 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f891bcde-3e1f-4ae7-b589-8d379dda7d68 None admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:38.457828 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f891bcde-3e1f-4ae7-b589-8d379dda7d68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:38.458201 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f891bcde-3e1f-4ae7-b589-8d379dda7d68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:38.459629 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f891bcde-3e1f-4ae7-b589-8d379dda7d68 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:38.472256 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 319/619] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:00:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:38.482183 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d0dbb6-b0c4-4bfe-80e8-68b640df2c0e None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:38.515100 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:38.551389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f891bcde-3e1f-4ae7-b589-8d379dda7d68 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:38.629252 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d0dbb6-b0c4-4bfe-80e8-68b640df2c0e None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:38.693175 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c235e4-de62-48b5-86f2-87e2212346f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:38.694216 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c235e4-de62-48b5-86f2-87e2212346f9 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:38.694635 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c235e4-de62-48b5-86f2-87e2212346f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:38.695056 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c235e4-de62-48b5-86f2-87e2212346f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:38.707338 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c235e4-de62-48b5-86f2-87e2212346f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:38.709011 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c235e4-de62-48b5-86f2-87e2212346f9 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:38.723057 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-99c235e4-de62-48b5-86f2-87e2212346f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:00:38.723057 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:38.723057 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:38.723057 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:38.723057 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:38.723057 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:38.723057 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:38.723057 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:38.723057 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:38.723057 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:38.723057 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:38.723057 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:38.723057 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 01:00:38.723057 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 01:00:38.723057 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 01:00:38.724154 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 01:00:38.724154 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:38.724154 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:38.724154 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:38.724154 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:38.724154 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:38.724154 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:38.724154 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:38.724154 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:38.724154 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:38.724154 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:38.724154 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:38.724154 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:38.724154 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:38.724154 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:38.724154 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:38.724154 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:38.725252 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 01:00:38.725252 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 01:00:38.725252 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 01:00:38.725252 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 01:00:38.725252 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 01:00:38.725252 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 01:00:38.725252 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:00:38.725252 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:38.726992 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 301/620] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 01:00:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:38.736156 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c235e4-de62-48b5-86f2-87e2212346f9 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:38.736304 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f891bcde-3e1f-4ae7-b589-8d379dda7d68 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:38.783869 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e831d60e-7c1e-48c8-848e-97c35cecee86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:38.784540 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e831d60e-7c1e-48c8-848e-97c35cecee86 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:38.784872 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e831d60e-7c1e-48c8-848e-97c35cecee86 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:38.785145 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e831d60e-7c1e-48c8-848e-97c35cecee86 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:38.808108 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e831d60e-7c1e-48c8-848e-97c35cecee86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:00:38.815441 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-e831d60e-7c1e-48c8-848e-97c35cecee86 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 01:00:38.815441 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:38.815441 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:38.815441 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:38.815441 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:38.815441 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:38.815441 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:38.815441 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:38.815441 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:38.815441 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:38.815441 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:38.815441 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:38.815441 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:00:38.815441 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:00:38.815441 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:00:38.815441 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:00:38.816405 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:00:38.816405 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:00:38.816405 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:00:38.816405 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:00:38.816405 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:38.816405 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:38.816405 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:38.816405 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:38.816405 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:38.816405 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:38.816405 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:38.816405 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:38.816405 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:38.816405 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:38.816405 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:38.816405 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:38.817212 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:38.817212 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:38.817212 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:38.817212 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:38.817212 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:00:38.817212 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:00:38.817212 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:00:38.817212 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:00:38.817212 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:00:38.817212 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:00:38.817212 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 01:00:38.817212 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:38.820362 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 320/621] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 8 01:00:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 01:00:38.824906 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e831d60e-7c1e-48c8-848e-97c35cecee86 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:38.860496 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:38.920650 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:38.931373 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c235e4-de62-48b5-86f2-87e2212346f9 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:38.995134 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e831d60e-7c1e-48c8-848e-97c35cecee86 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:39.021399 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3d2e9e-5f2b-4b30-b97b-664dc52925e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:39.022029 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d2e9e-5f2b-4b30-b97b-664dc52925e9 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:39.022311 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d2e9e-5f2b-4b30-b97b-664dc52925e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:39.022584 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d2e9e-5f2b-4b30-b97b-664dc52925e9 None admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:39.023286 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3d2e9e-5f2b-4b30-b97b-664dc52925e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:39.023594 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3d2e9e-5f2b-4b30-b97b-664dc52925e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:39.025085 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3d2e9e-5f2b-4b30-b97b-664dc52925e9 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:39.041577 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceb2a44-dd31-4d2a-a171-d815c6ef752c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:39.042191 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb2a44-dd31-4d2a-a171-d815c6ef752c None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:39.042469 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb2a44-dd31-4d2a-a171-d815c6ef752c None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:39.042848 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb2a44-dd31-4d2a-a171-d815c6ef752c None admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:39.043515 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceb2a44-dd31-4d2a-a171-d815c6ef752c None admin] RBAC: Adding query filter params (name=service) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:39.043866 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceb2a44-dd31-4d2a-a171-d815c6ef752c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:39.045266 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceb2a44-dd31-4d2a-a171-d815c6ef752c None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:39.056362 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 302/622] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 8 01:00:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:39.058016 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3d2e9e-5f2b-4b30-b97b-664dc52925e9 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:39.064461 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 321/623] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 8 01:00:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:39.069351 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceb2a44-dd31-4d2a-a171-d815c6ef752c None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:39.220886 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:39.238553 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3d2e9e-5f2b-4b30-b97b-664dc52925e9 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:39.247934 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceb2a44-dd31-4d2a-a171-d815c6ef752c None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:39.278802 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c71334-4dfb-4a67-aa03-bf45649a7280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:39.279575 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c71334-4dfb-4a67-aa03-bf45649a7280 None admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:39.279860 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c71334-4dfb-4a67-aa03-bf45649a7280 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:39.280162 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c71334-4dfb-4a67-aa03-bf45649a7280 None admin] PATH_INFO: `/v3/projects/78ad2a8fb5264fb4af935e039d28c8de/users/8add26872fe6479aa9d7de155a17195a/roles/609d73f56e6a48698397131168abe6df` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:39.293334 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b19bb86-2d9b-494f-8863-935c63e8d1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:39.294208 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19bb86-2d9b-494f-8863-935c63e8d1bc None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:39.294556 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19bb86-2d9b-494f-8863-935c63e8d1bc None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:39.294992 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19bb86-2d9b-494f-8863-935c63e8d1bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:39.297060 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c71334-4dfb-4a67-aa03-bf45649a7280 None admin] RBAC: Authorizing `identity:create_grant(project_id=78ad2a8fb5264fb4af935e039d28c8de, user_id=8add26872fe6479aa9d7de155a17195a, role_id=609d73f56e6a48698397131168abe6df)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:39.298640 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c71334-4dfb-4a67-aa03-bf45649a7280 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:39.309221 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b19bb86-2d9b-494f-8863-935c63e8d1bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:39.309744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:39.311715 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b19bb86-2d9b-494f-8863-935c63e8d1bc None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:39.315248 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 303/624] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 8 01:00:39 2022] PUT /identity/v3/projects/78ad2a8fb5264fb4af935e039d28c8de/users/8add26872fe6479aa9d7de155a17195a/roles/609d73f56e6a48698397131168abe6df => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:00:39.316778 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ecc28f-43d6-46eb-a4ce-976f375338c7 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:39.317938 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ecc28f-43d6-46eb-a4ce-976f375338c7 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:39.318457 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ecc28f-43d6-46eb-a4ce-976f375338c7 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:39.325245 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 304/625] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:00:39 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:00:39.327014 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b04c1e-0390-4b56-bb06-84ef9fa62e06 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:39.327299 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b04c1e-0390-4b56-bb06-84ef9fa62e06 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:39.327578 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b04c1e-0390-4b56-bb06-84ef9fa62e06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:39.331922 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-4b19bb86-2d9b-494f-8863-935c63e8d1bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:39.331922 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:39.331922 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:39.331922 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:39.331922 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:39.331922 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:39.331922 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:39.331922 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:39.331922 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:39.331922 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:39.331922 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:39.331922 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:39.331922 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:00:39.331922 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:00:39.331922 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:00:39.331922 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:00:39.333744 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:39.333744 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:39.333744 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:39.333744 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:39.333744 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:39.333744 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:39.333744 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:39.333744 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:39.333744 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:39.333744 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:39.333744 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:39.333744 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:39.333744 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:39.333744 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:39.333744 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:39.333744 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:39.333744 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:00:39.335416 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:00:39.335416 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:39.335416 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:39.335416 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 322/626] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:00:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 01:00:39.341811 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b19bb86-2d9b-494f-8863-935c63e8d1bc None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:39.347560 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-f7b04c1e-0390-4b56-bb06-84ef9fa62e06 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:00:39.348818 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-f7b04c1e-0390-4b56-bb06-84ef9fa62e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:00:39.428379 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 305/627] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:00:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 01:00:39.544604 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b19bb86-2d9b-494f-8863-935c63e8d1bc None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:39.577797 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:39.584867 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c819f7-54a1-481a-9f84-e2bcdeb04cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:39.585905 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c819f7-54a1-481a-9f84-e2bcdeb04cfa None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:39.586664 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c819f7-54a1-481a-9f84-e2bcdeb04cfa None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:39.587253 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c819f7-54a1-481a-9f84-e2bcdeb04cfa None admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:39.588481 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c819f7-54a1-481a-9f84-e2bcdeb04cfa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:39.588790 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c819f7-54a1-481a-9f84-e2bcdeb04cfa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:39.590752 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c819f7-54a1-481a-9f84-e2bcdeb04cfa None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:39.605814 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 323/628] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:00:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:39.611166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b04c1e-0390-4b56-bb06-84ef9fa62e06 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:39.726636 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b04c1e-0390-4b56-bb06-84ef9fa62e06 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:39.753096 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c819f7-54a1-481a-9f84-e2bcdeb04cfa None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:39.769595 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:39.777313 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c651fd-2c85-4dcc-bcc6-f71b9c690fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:39.778159 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c651fd-2c85-4dcc-bcc6-f71b9c690fe9 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:39.778558 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c651fd-2c85-4dcc-bcc6-f71b9c690fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:39.778837 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c651fd-2c85-4dcc-bcc6-f71b9c690fe9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:39.787242 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c651fd-2c85-4dcc-bcc6-f71b9c690fe9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:39.788892 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c651fd-2c85-4dcc-bcc6-f71b9c690fe9 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:39.803447 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-94c651fd-2c85-4dcc-bcc6-f71b9c690fe9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 01:00:39.803447 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:39.803447 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:39.803447 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:39.803447 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:39.803447 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:39.803447 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:39.803447 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:39.803447 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:39.803447 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:39.803447 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:39.803447 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:39.803447 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 01:00:39.803447 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 01:00:39.803447 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 01:00:39.803447 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 01:00:39.804531 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:39.804531 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:39.804531 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 01:00:39.804531 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:00:39.804531 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 01:00:39.804531 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:00:39.804531 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:39.804531 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:39.804531 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:39.804531 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:39.804531 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:39.804531 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:39.804531 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:39.804531 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:39.804531 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:39.804531 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:39.804531 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:39.804531 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:39.805478 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:39.805478 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:39.805478 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 01:00:39.805478 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 01:00:39.805478 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 01:00:39.805478 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 01:00:39.805478 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 01:00:39.805478 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 01:00:39.805478 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 01:00:39.805478 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:39.805478 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 306/629] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Sep 8 01:00:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:39.810091 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c651fd-2c85-4dcc-bcc6-f71b9c690fe9 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:39.901581 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c819f7-54a1-481a-9f84-e2bcdeb04cfa None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:39.932549 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c651fd-2c85-4dcc-bcc6-f71b9c690fe9 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:39.940642 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949bd9da-1615-456c-a962-70db835d0755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:39.941291 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949bd9da-1615-456c-a962-70db835d0755 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:39.941571 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949bd9da-1615-456c-a962-70db835d0755 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:39.941844 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949bd9da-1615-456c-a962-70db835d0755 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:39.954054 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949bd9da-1615-456c-a962-70db835d0755 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:00:39.959150 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-949bd9da-1615-456c-a962-70db835d0755 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 01:00:39.959150 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:39.959150 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:39.959150 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:39.959150 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:39.959150 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:39.959150 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:39.959150 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:39.959150 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:39.959150 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:39.959150 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:39.959150 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:39.959150 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:00:39.959150 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:00:39.959150 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:00:39.959150 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:00:39.960157 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:00:39.960157 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:00:39.960157 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:00:39.960157 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:00:39.960157 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:39.960157 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:39.960157 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:39.960157 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:39.960157 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:39.960157 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:39.960157 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:39.960157 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:39.960157 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:39.960157 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:39.960157 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:39.960157 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:39.961000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:39.961000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:39.961000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:39.961000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:39.961000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:00:39.961000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:00:39.961000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:00:39.961000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:00:39.961000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:00:39.961000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:00:39.961000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 01:00:39.961000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:39.961000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 324/630] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 8 01:00:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 01:00:39.962172 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b6da74-accd-4510-82a2-8aec9249ec1c None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:39.962324 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b6da74-accd-4510-82a2-8aec9249ec1c None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:39.962476 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b6da74-accd-4510-82a2-8aec9249ec1c None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:39.965558 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 325/631] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:00:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:00:39.967001 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b6da74-accd-4510-82a2-8aec9249ec1c None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:39.979951 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad2b0e9-4e8d-4c69-b5e1-2e24576ec4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:39.981251 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2b0e9-4e8d-4c69-b5e1-2e24576ec4a8 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:39.981615 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2b0e9-4e8d-4c69-b5e1-2e24576ec4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:39.984156 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2b0e9-4e8d-4c69-b5e1-2e24576ec4a8 None admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:39.984156 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad2b0e9-4e8d-4c69-b5e1-2e24576ec4a8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:39.984156 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad2b0e9-4e8d-4c69-b5e1-2e24576ec4a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:39.984647 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad2b0e9-4e8d-4c69-b5e1-2e24576ec4a8 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:40.004780 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 307/632] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Sep 8 01:00:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:40.006293 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ff916-996a-4a17-b644-1387bad3ff49 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:40.006584 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ff916-996a-4a17-b644-1387bad3ff49 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:40.006859 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ff916-996a-4a17-b644-1387bad3ff49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:40.022955 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:40.028579 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-902ff916-996a-4a17-b644-1387bad3ff49 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:00:40.031017 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-902ff916-996a-4a17-b644-1387bad3ff49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:00:40.078660 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b6da74-accd-4510-82a2-8aec9249ec1c None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:40.080421 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:40.108454 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 308/633] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:00:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 01:00:40.108454 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ff916-996a-4a17-b644-1387bad3ff49 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:40.124317 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cadec5d-7199-4754-800a-9044e64b21e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:40.125425 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cadec5d-7199-4754-800a-9044e64b21e2 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:40.126241 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cadec5d-7199-4754-800a-9044e64b21e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:40.126679 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cadec5d-7199-4754-800a-9044e64b21e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:40.127508 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cadec5d-7199-4754-800a-9044e64b21e2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:40.127981 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cadec5d-7199-4754-800a-9044e64b21e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:40.130331 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cadec5d-7199-4754-800a-9044e64b21e2 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:40.146101 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 326/634] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 8 01:00:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:40.146101 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd1989-b4f1-4a82-a176-07e5603fe856 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:40.146101 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd1989-b4f1-4a82-a176-07e5603fe856 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:40.146432 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd1989-b4f1-4a82-a176-07e5603fe856 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:40.148665 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 327/635] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:00:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:00:40.152975 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fd1989-b4f1-4a82-a176-07e5603fe856 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:40.220453 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ff916-996a-4a17-b644-1387bad3ff49 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:40.256619 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ac8d3-dbf5-4720-b14a-fc9a77e16ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:40.257227 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ac8d3-dbf5-4720-b14a-fc9a77e16ac3 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:40.257510 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ac8d3-dbf5-4720-b14a-fc9a77e16ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:40.257846 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ac8d3-dbf5-4720-b14a-fc9a77e16ac3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:40.277071 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fd1989-b4f1-4a82-a176-07e5603fe856 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:40.294044 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:40.307925 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:40.321779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e002ebc5-e15b-487e-96ad-72dbf90986c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:40.322753 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e002ebc5-e15b-487e-96ad-72dbf90986c5 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:40.323233 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e002ebc5-e15b-487e-96ad-72dbf90986c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:40.324578 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e002ebc5-e15b-487e-96ad-72dbf90986c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:40.331874 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874ac8d3-dbf5-4720-b14a-fc9a77e16ac3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:40.333494 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874ac8d3-dbf5-4720-b14a-fc9a77e16ac3 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:40.341055 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e002ebc5-e15b-487e-96ad-72dbf90986c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:40.343352 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e002ebc5-e15b-487e-96ad-72dbf90986c5 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:40.363047 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-e002ebc5-e15b-487e-96ad-72dbf90986c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:40.363047 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:40.363047 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:40.363047 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:40.363047 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:40.363047 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:40.363047 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:40.363047 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:40.363047 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:40.363047 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:40.363047 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:40.363047 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:40.363047 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:00:40.363047 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:00:40.363047 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:00:40.363047 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:00:40.364050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:40.364050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:40.364050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:40.364050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:40.364050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:40.364050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:40.364050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:40.364050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:40.364050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:40.364050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:40.364050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:40.364050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:40.364050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:40.364050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:40.364050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:40.364050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:40.364050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:00:40.364977 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:00:40.364977 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:40.364977 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:40.367680 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 328/636] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:00:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:40.371925 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe0402-22a4-47de-a71a-372b8893859e None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:40.372241 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe0402-22a4-47de-a71a-372b8893859e None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:40.372521 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe0402-22a4-47de-a71a-372b8893859e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:40.394203 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-d2fe0402-22a4-47de-a71a-372b8893859e None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:00:40.395401 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-d2fe0402-22a4-47de-a71a-372b8893859e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:00:40.458285 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 309/637] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 01:00:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 01:00:40.459138 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ac8d3-dbf5-4720-b14a-fc9a77e16ac3 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:40.472768 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 329/638] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:00:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 01:00:40.473850 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fe0402-22a4-47de-a71a-372b8893859e None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:40.559449 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:40.586378 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:40.594474 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ac8d3-dbf5-4720-b14a-fc9a77e16ac3 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:40.595122 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fe0402-22a4-47de-a71a-372b8893859e None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:40.641522 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b111e-19b3-4208-b8c8-6fb802eeca62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:40.643013 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b111e-19b3-4208-b8c8-6fb802eeca62 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:40.643581 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b111e-19b3-4208-b8c8-6fb802eeca62 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:40.644039 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b111e-19b3-4208-b8c8-6fb802eeca62 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:40.653827 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7edb9a-070c-4a97-a002-2b6a789c8d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:40.654509 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7edb9a-070c-4a97-a002-2b6a789c8d69 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:40.654857 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7edb9a-070c-4a97-a002-2b6a789c8d69 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:40.655206 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7edb9a-070c-4a97-a002-2b6a789c8d69 None admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:40.655901 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7edb9a-070c-4a97-a002-2b6a789c8d69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:40.656325 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7edb9a-070c-4a97-a002-2b6a789c8d69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:40.657761 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7edb9a-070c-4a97-a002-2b6a789c8d69 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:40.659952 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8b111e-19b3-4208-b8c8-6fb802eeca62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:00:40.666130 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-6d8b111e-19b3-4208-b8c8-6fb802eeca62 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 01:00:40.666130 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:40.666130 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:40.666130 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:40.666130 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:40.666130 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:40.666130 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:40.666130 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:40.666130 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:40.666130 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:40.666130 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:40.666130 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:40.666130 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:00:40.666130 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:00:40.666130 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:00:40.666130 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:00:40.667107 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:00:40.667107 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:00:40.667107 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:00:40.667107 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:00:40.667107 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:40.667107 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:40.667107 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:40.667107 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:40.667107 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:40.667107 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:40.667107 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:40.667107 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:40.667107 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:40.667107 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:40.667107 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:40.667107 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:40.668757 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:40.668757 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:40.668757 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:40.668757 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:40.668757 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:00:40.668757 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:00:40.668757 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:00:40.668757 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:00:40.668757 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:00:40.668757 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:00:40.668757 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 01:00:40.668757 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:40.676301 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 310/639] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:00:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:40.676301 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7edb9a-070c-4a97-a002-2b6a789c8d69 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:40.676790 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 330/640] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 8 01:00:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:40.677853 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b111e-19b3-4208-b8c8-6fb802eeca62 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:40.797022 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7edb9a-070c-4a97-a002-2b6a789c8d69 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:40.798195 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b111e-19b3-4208-b8c8-6fb802eeca62 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:40.834148 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aec7cb-f5e0-4292-8d16-82c40457249f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:40.834755 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aec7cb-f5e0-4292-8d16-82c40457249f None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:40.835034 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aec7cb-f5e0-4292-8d16-82c40457249f None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:40.835306 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aec7cb-f5e0-4292-8d16-82c40457249f None admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:40.835745 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bad9ec-c814-4f68-bdf2-eb8cba870fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:40.835966 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aec7cb-f5e0-4292-8d16-82c40457249f None admin] RBAC: Adding query filter params (name=service) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:40.836297 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aec7cb-f5e0-4292-8d16-82c40457249f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:40.836937 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bad9ec-c814-4f68-bdf2-eb8cba870fd8 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:40.837380 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bad9ec-c814-4f68-bdf2-eb8cba870fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:40.837666 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aec7cb-f5e0-4292-8d16-82c40457249f None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:40.837958 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bad9ec-c814-4f68-bdf2-eb8cba870fd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:40.850706 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:40.850875 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 331/641] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 8 01:00:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:40.850875 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aec7cb-f5e0-4292-8d16-82c40457249f None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:40.853070 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bad9ec-c814-4f68-bdf2-eb8cba870fd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:40.854907 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bad9ec-c814-4f68-bdf2-eb8cba870fd8 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:40.869803 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:40.871740 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-a5bad9ec-c814-4f68-bdf2-eb8cba870fd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:40.871740 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:40.871740 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:40.871740 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:40.871740 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:40.871740 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:40.871740 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:40.871740 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:40.871740 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:40.871740 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:40.871740 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:40.871740 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:40.871740 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:00:40.871740 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:00:40.871740 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:00:40.871740 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:00:40.873437 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:40.873437 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:40.873437 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:40.873437 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:40.873437 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:40.873437 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:40.873437 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:40.873437 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:40.873437 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:40.873437 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:40.873437 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:40.873437 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:40.873437 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:40.873437 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:40.873437 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:40.873437 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:40.873437 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:00:40.874332 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:00:40.874332 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:40.874332 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:40.875784 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 311/642] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:00:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:40.876794 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bad9ec-c814-4f68-bdf2-eb8cba870fd8 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:40.959584 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aec7cb-f5e0-4292-8d16-82c40457249f None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:40.984365 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bad9ec-c814-4f68-bdf2-eb8cba870fd8 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:40.994624 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240c1f09-62ce-45fd-9b89-b98a9b063dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:40.995247 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c1f09-62ce-45fd-9b89-b98a9b063dc2 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:40.995522 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c1f09-62ce-45fd-9b89-b98a9b063dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:40.995803 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c1f09-62ce-45fd-9b89-b98a9b063dc2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:41.004800 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240c1f09-62ce-45fd-9b89-b98a9b063dc2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:41.006246 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240c1f09-62ce-45fd-9b89-b98a9b063dc2 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:41.018102 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-240c1f09-62ce-45fd-9b89-b98a9b063dc2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 08 01:00:41.018102 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:41.018102 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:41.018102 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:41.018102 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:41.018102 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:41.018102 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:41.018102 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:41.018102 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:41.018102 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:41.018102 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:41.018102 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:41.018102 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 01:00:41.018102 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 01:00:41.018102 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 01:00:41.018102 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 01:00:41.019283 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:41.019283 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:41.019283 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 01:00:41.019283 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:00:41.019283 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 01:00:41.019283 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:00:41.019283 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:41.019283 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:41.019283 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:41.019283 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:41.019283 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:41.019283 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:41.019283 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:41.019283 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:41.019283 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:41.019283 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:41.019283 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:41.019283 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:41.020497 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:41.020497 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:41.020497 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 01:00:41.020497 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 01:00:41.020497 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 01:00:41.020497 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 01:00:41.020497 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 01:00:41.020497 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 01:00:41.020497 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 08 01:00:41.020497 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:41.020497 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 332/643] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 01:00:40 2022] GET /identity/v3/users/glance => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:41.020497 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240c1f09-62ce-45fd-9b89-b98a9b063dc2 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:41.027634 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead1ca8a-7dcc-4064-a286-370b87dd8289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:41.028380 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead1ca8a-7dcc-4064-a286-370b87dd8289 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:41.028665 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead1ca8a-7dcc-4064-a286-370b87dd8289 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:41.028942 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead1ca8a-7dcc-4064-a286-370b87dd8289 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:41.039348 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead1ca8a-7dcc-4064-a286-370b87dd8289 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:00:41.045204 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-ead1ca8a-7dcc-4064-a286-370b87dd8289 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 01:00:41.045204 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:41.045204 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:41.045204 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:41.045204 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:41.045204 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:41.045204 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:41.045204 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:41.045204 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:41.045204 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:41.045204 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:41.045204 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:41.045204 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:00:41.045204 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:00:41.045204 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:00:41.045204 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:00:41.046107 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:00:41.046107 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:00:41.046107 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:00:41.046107 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:00:41.046107 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:41.046107 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:41.046107 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:41.046107 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:41.046107 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:41.046107 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:41.046107 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:41.046107 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:41.046107 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:41.046107 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:41.046107 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:41.046107 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:41.049199 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:41.049199 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:41.049199 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:41.049199 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:41.049199 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:00:41.049199 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:00:41.049199 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:00:41.049199 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:00:41.049199 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:00:41.049199 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:00:41.049199 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 01:00:41.049199 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:41.049199 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 312/644] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 8 01:00:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:41.049199 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead1ca8a-7dcc-4064-a286-370b87dd8289 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:41.101188 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:41.106756 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:41.130473 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240c1f09-62ce-45fd-9b89-b98a9b063dc2 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:41.159046 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead1ca8a-7dcc-4064-a286-370b87dd8289 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:41.165062 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf7f789-0fe8-42e7-b2a0-adba2a8f1827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:41.165975 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf7f789-0fe8-42e7-b2a0-adba2a8f1827 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:41.166623 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf7f789-0fe8-42e7-b2a0-adba2a8f1827 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:41.167058 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf7f789-0fe8-42e7-b2a0-adba2a8f1827 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:41.179286 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf7f789-0fe8-42e7-b2a0-adba2a8f1827 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:41.180969 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf7f789-0fe8-42e7-b2a0-adba2a8f1827 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:41.194739 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e58c46a-7555-41b5-9abd-1bf6a3392716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:41.195351 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58c46a-7555-41b5-9abd-1bf6a3392716 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:41.195633 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58c46a-7555-41b5-9abd-1bf6a3392716 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:41.195734 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-acf7f789-0fe8-42e7-b2a0-adba2a8f1827 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:41.195734 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:41.195734 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:41.195734 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:41.195734 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:41.195734 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:41.195734 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:41.195734 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:41.195734 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:41.195734 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:41.195734 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:41.195734 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:41.195734 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:00:41.195734 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:00:41.195734 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:00:41.195734 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:00:41.196626 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:41.196626 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:41.196626 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:41.196626 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:41.196626 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:41.196626 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:41.196626 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:41.196626 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:41.196626 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:41.196626 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:41.196626 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:41.196626 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:41.196626 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:41.196626 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:41.196626 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:41.196626 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:41.196626 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:00:41.197528 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:00:41.197528 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:41.197528 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:41.197694 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58c46a-7555-41b5-9abd-1bf6a3392716 None admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:41.197694 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e58c46a-7555-41b5-9abd-1bf6a3392716 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:41.197694 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e58c46a-7555-41b5-9abd-1bf6a3392716 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:41.198365 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e58c46a-7555-41b5-9abd-1bf6a3392716 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:41.202036 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 333/645] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:00:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:41.202833 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf7f789-0fe8-42e7-b2a0-adba2a8f1827 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:41.216676 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 313/646] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:00:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:41.216676 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e58c46a-7555-41b5-9abd-1bf6a3392716 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:41.313301 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf7f789-0fe8-42e7-b2a0-adba2a8f1827 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:41.313669 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e58c46a-7555-41b5-9abd-1bf6a3392716 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:41.325409 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:41.343282 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:41.351255 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f98582-67b5-42e1-992d-b1bb82647fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:41.351504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa4b294-7875-406e-820b-98efae38d62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:41.351809 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f98582-67b5-42e1-992d-b1bb82647fd5 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:41.352047 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa4b294-7875-406e-820b-98efae38d62e None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:41.352269 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f98582-67b5-42e1-992d-b1bb82647fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:41.352358 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa4b294-7875-406e-820b-98efae38d62e None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:41.352430 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f98582-67b5-42e1-992d-b1bb82647fd5 None admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:41.352599 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa4b294-7875-406e-820b-98efae38d62e None admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:41.353090 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f98582-67b5-42e1-992d-b1bb82647fd5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:41.353247 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa4b294-7875-406e-820b-98efae38d62e None admin] RBAC: Adding query filter params (name=service) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:41.353394 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f98582-67b5-42e1-992d-b1bb82647fd5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:41.353537 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa4b294-7875-406e-820b-98efae38d62e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:41.354811 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f98582-67b5-42e1-992d-b1bb82647fd5 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:41.354909 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa4b294-7875-406e-820b-98efae38d62e None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:41.368439 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 314/647] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 8 01:00:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:41.368439 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa4b294-7875-406e-820b-98efae38d62e None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:41.383331 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 334/648] 10.222.0.42 () {56 vars in 1137 bytes} [Thu Sep 8 01:00:41 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:41.383331 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f98582-67b5-42e1-992d-b1bb82647fd5 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:41.468214 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa4b294-7875-406e-820b-98efae38d62e None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:41.492550 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f98582-67b5-42e1-992d-b1bb82647fd5 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:41.504657 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d21c4fd-61bc-4f43-923c-4c8cb871edf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:41.505516 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d21c4fd-61bc-4f43-923c-4c8cb871edf3 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:41.506052 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d21c4fd-61bc-4f43-923c-4c8cb871edf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:41.506577 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d21c4fd-61bc-4f43-923c-4c8cb871edf3 None admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:41.507358 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d21c4fd-61bc-4f43-923c-4c8cb871edf3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:41.507663 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d21c4fd-61bc-4f43-923c-4c8cb871edf3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:41.509082 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d21c4fd-61bc-4f43-923c-4c8cb871edf3 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:41.526265 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 315/649] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:00:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:41.526265 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d21c4fd-61bc-4f43-923c-4c8cb871edf3 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:41.530380 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19c7bf3-0d75-4b4a-a0cb-ea4cc10b0f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:41.531830 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c7bf3-0d75-4b4a-a0cb-ea4cc10b0f99 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:41.532107 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c7bf3-0d75-4b4a-a0cb-ea4cc10b0f99 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:41.532401 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c7bf3-0d75-4b4a-a0cb-ea4cc10b0f99 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:41.540284 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19c7bf3-0d75-4b4a-a0cb-ea4cc10b0f99 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:41.541730 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19c7bf3-0d75-4b4a-a0cb-ea4cc10b0f99 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:41.553369 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-b19c7bf3-0d75-4b4a-a0cb-ea4cc10b0f99 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:00:41.553369 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:41.553369 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:41.553369 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:41.553369 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:41.553369 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:41.553369 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:41.553369 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:41.553369 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:41.553369 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:41.553369 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:41.553369 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:41.553369 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 01:00:41.553369 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 01:00:41.553369 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 01:00:41.554273 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 01:00:41.554273 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:41.554273 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:41.554273 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:41.554273 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:41.554273 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:41.554273 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:41.554273 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:41.554273 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:41.554273 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:41.554273 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:41.554273 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:41.554273 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:41.554273 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:41.554273 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:41.554273 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:41.554273 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:41.555147 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 01:00:41.555147 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 01:00:41.555147 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 01:00:41.555147 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 01:00:41.555147 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 01:00:41.555147 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 01:00:41.555147 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:00:41.555147 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:41.555147 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 335/650] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 01:00:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:41.556276 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19c7bf3-0d75-4b4a-a0cb-ea4cc10b0f99 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:41.565508 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:41.607185 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:41.648364 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d21c4fd-61bc-4f43-923c-4c8cb871edf3 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:41.674309 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19c7bf3-0d75-4b4a-a0cb-ea4cc10b0f99 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:41.685655 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575c857a-6153-48e7-95df-308d8269b301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:41.686371 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c857a-6153-48e7-95df-308d8269b301 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:41.686727 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c857a-6153-48e7-95df-308d8269b301 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:41.687078 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c857a-6153-48e7-95df-308d8269b301 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:41.697386 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575c857a-6153-48e7-95df-308d8269b301 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:41.698985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575c857a-6153-48e7-95df-308d8269b301 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:41.709885 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e00221-d4f6-41f9-8cb0-6e4e456c56b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:41.710487 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e00221-d4f6-41f9-8cb0-6e4e456c56b8 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:41.710763 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e00221-d4f6-41f9-8cb0-6e4e456c56b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:41.711046 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e00221-d4f6-41f9-8cb0-6e4e456c56b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:41.714232 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-575c857a-6153-48e7-95df-308d8269b301 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:41.714232 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:41.714232 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:41.714232 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:41.714232 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:41.714232 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:41.714232 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:41.714232 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:41.714232 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:41.714232 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:41.714232 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:41.714232 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:41.714232 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:00:41.714232 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:00:41.714232 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:00:41.714232 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:00:41.715737 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:41.715737 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:41.715737 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:41.715737 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:41.715737 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:41.715737 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:41.715737 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:41.715737 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:41.715737 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:41.715737 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:41.715737 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:41.715737 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:41.715737 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:41.715737 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:41.715737 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:41.715737 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:41.715737 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:00:41.716654 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:00:41.716654 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:41.716654 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:41.716654 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 316/651] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:00:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:41.716875 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575c857a-6153-48e7-95df-308d8269b301 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:41.785340 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e00221-d4f6-41f9-8cb0-6e4e456c56b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:41.786817 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e00221-d4f6-41f9-8cb0-6e4e456c56b8 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:41.802788 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:41.819169 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575c857a-6153-48e7-95df-308d8269b301 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:41.831629 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:41.852495 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c49014-41f3-4d65-90c6-94e2d9de48b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:41.853113 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c49014-41f3-4d65-90c6-94e2d9de48b2 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:41.853391 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c49014-41f3-4d65-90c6-94e2d9de48b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:41.853662 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c49014-41f3-4d65-90c6-94e2d9de48b2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:41.862164 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c49014-41f3-4d65-90c6-94e2d9de48b2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:41.863939 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c49014-41f3-4d65-90c6-94e2d9de48b2 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:41.875608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-00c49014-41f3-4d65-90c6-94e2d9de48b2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 01:00:41.875608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:41.875608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:41.875608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:41.875608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:41.875608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:41.875608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:41.875608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:41.875608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:41.875608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:41.875608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:41.875608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:41.875608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 01:00:41.875608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 01:00:41.875608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 01:00:41.875608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 01:00:41.876694 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:41.876694 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:41.876694 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 01:00:41.876694 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:00:41.876694 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 01:00:41.876694 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:00:41.876694 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:41.876694 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:41.876694 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:41.876694 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:41.876694 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:41.876694 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:41.876694 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:41.876694 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:41.876694 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:41.876694 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:41.876694 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:41.876694 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:41.878013 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:41.878013 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:41.878013 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 01:00:41.878013 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 01:00:41.878013 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 01:00:41.878013 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 01:00:41.878013 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 01:00:41.878013 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 01:00:41.878013 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 01:00:41.878013 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:41.878013 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 317/652] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 01:00:41 2022] GET /identity/v3/users/cinder => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:41.878013 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c49014-41f3-4d65-90c6-94e2d9de48b2 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:41.892044 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 336/653] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 01:00:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 01:00:41.892900 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e00221-d4f6-41f9-8cb0-6e4e456c56b8 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:41.981120 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c49014-41f3-4d65-90c6-94e2d9de48b2 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:41.990661 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e00221-d4f6-41f9-8cb0-6e4e456c56b8 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:42.014535 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6495a3e5-e61a-4406-8ebd-b654fee57916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:42.015136 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495a3e5-e61a-4406-8ebd-b654fee57916 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:42.015413 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495a3e5-e61a-4406-8ebd-b654fee57916 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:42.015686 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495a3e5-e61a-4406-8ebd-b654fee57916 None admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:42.016400 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6495a3e5-e61a-4406-8ebd-b654fee57916 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:42.016702 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6495a3e5-e61a-4406-8ebd-b654fee57916 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:42.018142 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6495a3e5-e61a-4406-8ebd-b654fee57916 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:42.024543 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e79eb-30ba-4dfc-9cb1-99b31246737d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:42.025144 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e79eb-30ba-4dfc-9cb1-99b31246737d None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:42.025424 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e79eb-30ba-4dfc-9cb1-99b31246737d None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:42.025698 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e79eb-30ba-4dfc-9cb1-99b31246737d None admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:42.026334 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36e79eb-30ba-4dfc-9cb1-99b31246737d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:42.026636 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36e79eb-30ba-4dfc-9cb1-99b31246737d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:42.028221 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36e79eb-30ba-4dfc-9cb1-99b31246737d None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:42.029950 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 318/654] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 8 01:00:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:42.031106 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941eef57-57b1-4981-957c-4538360277f0 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:42.031388 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941eef57-57b1-4981-957c-4538360277f0 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:42.031661 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941eef57-57b1-4981-957c-4538360277f0 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:42.036254 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 319/655] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:00:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:00:42.036254 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941eef57-57b1-4981-957c-4538360277f0 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:42.039868 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:42.044319 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 337/656] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:00:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:42.044319 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e79eb-30ba-4dfc-9cb1-99b31246737d None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:42.051448 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:42.139430 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941eef57-57b1-4981-957c-4538360277f0 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:42.141775 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e79eb-30ba-4dfc-9cb1-99b31246737d None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:42.174909 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bfec91-6930-468e-b98b-04c8e1280580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:42.175499 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bfec91-6930-468e-b98b-04c8e1280580 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:42.175775 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bfec91-6930-468e-b98b-04c8e1280580 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:42.176043 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bfec91-6930-468e-b98b-04c8e1280580 None admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:42.176783 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8867cc77-a2d6-424c-85d1-26b5d6da562e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:42.176900 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bfec91-6930-468e-b98b-04c8e1280580 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:42.177081 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bfec91-6930-468e-b98b-04c8e1280580 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:42.177420 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867cc77-a2d6-424c-85d1-26b5d6da562e None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:42.177696 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867cc77-a2d6-424c-85d1-26b5d6da562e None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:42.177962 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867cc77-a2d6-424c-85d1-26b5d6da562e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:42.178506 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bfec91-6930-468e-b98b-04c8e1280580 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:42.188435 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8867cc77-a2d6-424c-85d1-26b5d6da562e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:42.189855 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8867cc77-a2d6-424c-85d1-26b5d6da562e None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:42.200198 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 320/657] 10.222.0.42 () {56 vars in 1137 bytes} [Thu Sep 8 01:00:42 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:42.200198 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bfec91-6930-468e-b98b-04c8e1280580 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:42.204002 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-8867cc77-a2d6-424c-85d1-26b5d6da562e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:42.204002 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:42.204002 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:42.204002 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:42.204002 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:42.204002 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:42.204002 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:42.204002 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:42.204002 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:42.204002 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:42.204002 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:42.204002 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:42.204002 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:00:42.204002 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:00:42.204002 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:00:42.204002 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:00:42.204993 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:42.204993 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:42.204993 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:42.204993 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:42.204993 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:42.204993 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:42.204993 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:42.204993 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:42.204993 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:42.204993 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:42.204993 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:42.204993 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:42.204993 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:42.204993 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:42.204993 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:42.204993 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:42.204993 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:00:42.205909 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:00:42.205909 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:42.205909 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:42.205909 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 338/658] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:00:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:42.206545 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f464538-c4b0-4cfb-a349-203979e0eed9 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:42.206545 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f464538-c4b0-4cfb-a349-203979e0eed9 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:42.206757 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f464538-c4b0-4cfb-a349-203979e0eed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:42.225545 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-2f464538-c4b0-4cfb-a349-203979e0eed9 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:00:42.226727 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-2f464538-c4b0-4cfb-a349-203979e0eed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:00:42.259585 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:42.280422 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:42.294982 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 339/659] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:00:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 01:00:42.295789 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f464538-c4b0-4cfb-a349-203979e0eed9 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:42.303646 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bfec91-6930-468e-b98b-04c8e1280580 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:42.341172 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b23ef93-16f1-457c-9d43-6a1b9368731d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:42.342375 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b23ef93-16f1-457c-9d43-6a1b9368731d None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:42.342852 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b23ef93-16f1-457c-9d43-6a1b9368731d None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:42.343318 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b23ef93-16f1-457c-9d43-6a1b9368731d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:42.344275 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b23ef93-16f1-457c-9d43-6a1b9368731d None admin] RBAC: Adding query filter params (role.id=609d73f56e6a48698397131168abe6df, scope.project.id=78ad2a8fb5264fb4af935e039d28c8de, user.id=ae643cfdcb164aa9b474747cd21aa116) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:42.344817 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b23ef93-16f1-457c-9d43-6a1b9368731d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:42.346356 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b23ef93-16f1-457c-9d43-6a1b9368731d None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:42.352161 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 321/660] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 8 01:00:42 2022] GET /identity/v3/role_assignments?user.id=ae643cfdcb164aa9b474747cd21aa116&scope.project.id=78ad2a8fb5264fb4af935e039d28c8de&role.id=609d73f56e6a48698397131168abe6df => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:42.352954 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b23ef93-16f1-457c-9d43-6a1b9368731d None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:42.424299 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f464538-c4b0-4cfb-a349-203979e0eed9 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:42.469133 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1d23f0-63a7-4c29-9c05-fa9f0817eb88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:42.470159 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d23f0-63a7-4c29-9c05-fa9f0817eb88 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:42.470637 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d23f0-63a7-4c29-9c05-fa9f0817eb88 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:42.471130 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d23f0-63a7-4c29-9c05-fa9f0817eb88 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:42.479862 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1d23f0-63a7-4c29-9c05-fa9f0817eb88 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:42.482262 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1d23f0-63a7-4c29-9c05-fa9f0817eb88 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:42.497691 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-ee1d23f0-63a7-4c29-9c05-fa9f0817eb88 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 08 01:00:42.497691 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:42.497691 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:42.497691 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:42.497691 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:42.497691 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:42.497691 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:42.497691 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:42.497691 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:42.497691 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:42.497691 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:42.497691 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:42.497691 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 01:00:42.497691 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 01:00:42.497691 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 01:00:42.497691 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 01:00:42.498726 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:42.498726 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:42.498726 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 01:00:42.498726 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:00:42.498726 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 01:00:42.498726 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:00:42.498726 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:42.498726 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:42.498726 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:42.498726 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:42.498726 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:42.498726 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:42.498726 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:42.498726 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:42.498726 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:42.498726 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:42.498726 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:42.498726 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:42.500428 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:42.500428 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:42.500428 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 01:00:42.500428 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 01:00:42.500428 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 01:00:42.500428 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 01:00:42.500428 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 01:00:42.500428 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 01:00:42.500428 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 08 01:00:42.500428 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:42.500428 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 340/661] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 8 01:00:42 2022] GET /identity/v3/users/neutron => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:42.500428 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1d23f0-63a7-4c29-9c05-fa9f0817eb88 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:42.521950 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b23ef93-16f1-457c-9d43-6a1b9368731d None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:42.557728 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:42.558140 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:42.576524 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ab0f59-5f3b-4780-b0ac-15e74af0a6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:42.577497 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab0f59-5f3b-4780-b0ac-15e74af0a6fa None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:42.577928 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab0f59-5f3b-4780-b0ac-15e74af0a6fa None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:42.578346 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab0f59-5f3b-4780-b0ac-15e74af0a6fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:42.616488 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1d23f0-63a7-4c29-9c05-fa9f0817eb88 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:42.657236 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3d20d-5182-420c-a339-8604957caa68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:42.657837 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3d20d-5182-420c-a339-8604957caa68 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:42.658114 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3d20d-5182-420c-a339-8604957caa68 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:42.658376 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3d20d-5182-420c-a339-8604957caa68 None admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:42.659043 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f3d20d-5182-420c-a339-8604957caa68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:42.659342 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f3d20d-5182-420c-a339-8604957caa68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:42.659632 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ab0f59-5f3b-4780-b0ac-15e74af0a6fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:42.660828 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f3d20d-5182-420c-a339-8604957caa68 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:42.661120 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ab0f59-5f3b-4780-b0ac-15e74af0a6fa None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:42.673823 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 341/662] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:00:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:42.674699 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3d20d-5182-420c-a339-8604957caa68 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:42.765983 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 322/663] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 01:00:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 01:00:42.766605 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ab0f59-5f3b-4780-b0ac-15e74af0a6fa None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:42.787384 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3d20d-5182-420c-a339-8604957caa68 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:42.794479 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:42.799260 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:42.827770 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d07518-56ef-45e5-8412-d12d7a571387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:42.828435 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d07518-56ef-45e5-8412-d12d7a571387 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:42.828741 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d07518-56ef-45e5-8412-d12d7a571387 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:42.829021 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d07518-56ef-45e5-8412-d12d7a571387 None admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:42.829717 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d07518-56ef-45e5-8412-d12d7a571387 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:42.830028 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d07518-56ef-45e5-8412-d12d7a571387 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:42.831472 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d07518-56ef-45e5-8412-d12d7a571387 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:42.854105 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 342/664] 10.222.0.42 () {56 vars in 1139 bytes} [Thu Sep 8 01:00:42 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:42.854105 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d07518-56ef-45e5-8412-d12d7a571387 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:42.877975 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ab0f59-5f3b-4780-b0ac-15e74af0a6fa None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:42.912650 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff6bb5e-c66a-4ea3-ab78-aa913f49f0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:42.913283 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff6bb5e-c66a-4ea3-ab78-aa913f49f0bf None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:42.913569 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff6bb5e-c66a-4ea3-ab78-aa913f49f0bf None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:42.913843 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff6bb5e-c66a-4ea3-ab78-aa913f49f0bf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:42.924262 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff6bb5e-c66a-4ea3-ab78-aa913f49f0bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:00:42.929281 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-5ff6bb5e-c66a-4ea3-ab78-aa913f49f0bf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 01:00:42.929281 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:42.929281 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:42.929281 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:42.929281 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:42.929281 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:42.929281 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:42.929281 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:42.929281 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:42.929281 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:42.929281 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:42.929281 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:42.929281 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:00:42.929281 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:00:42.929281 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:00:42.929281 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:00:42.930238 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:00:42.930238 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:00:42.930238 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:00:42.930238 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:00:42.930238 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:42.930238 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:42.930238 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:42.930238 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:42.930238 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:42.930238 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:42.930238 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:42.930238 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:42.930238 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:42.930238 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:42.930238 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:42.930238 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:42.933235 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:42.933235 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:42.933235 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:42.933235 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:42.933235 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:00:42.933235 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:00:42.933235 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:00:42.933235 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:00:42.933235 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:00:42.933235 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:00:42.933235 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 01:00:42.933235 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:42.933235 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 323/665] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 8 01:00:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:42.933235 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff6bb5e-c66a-4ea3-ab78-aa913f49f0bf None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:42.962060 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d07518-56ef-45e5-8412-d12d7a571387 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:43.001492 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a6a787-7418-4bc7-a0b9-ec770a472c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:43.002126 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a6a787-7418-4bc7-a0b9-ec770a472c07 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:43.002410 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a6a787-7418-4bc7-a0b9-ec770a472c07 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:43.002685 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a6a787-7418-4bc7-a0b9-ec770a472c07 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:43.010458 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a6a787-7418-4bc7-a0b9-ec770a472c07 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:43.013045 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a6a787-7418-4bc7-a0b9-ec770a472c07 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:43.022723 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:43.025694 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-53a6a787-7418-4bc7-a0b9-ec770a472c07 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:00:43.025694 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:43.025694 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:43.025694 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:43.025694 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:43.025694 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:43.025694 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:43.025694 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:43.025694 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:43.025694 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:43.025694 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:43.025694 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:43.025694 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 01:00:43.025694 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 01:00:43.025694 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 01:00:43.026580 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 01:00:43.026580 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:43.026580 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:43.026580 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:43.026580 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:43.026580 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:43.026580 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:43.026580 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:43.026580 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:43.026580 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:43.026580 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:43.026580 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:43.026580 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:43.026580 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:43.026580 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:43.026580 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:43.026580 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:43.028174 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 01:00:43.028174 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 01:00:43.028174 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 01:00:43.028174 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 01:00:43.028174 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 01:00:43.028174 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 01:00:43.028174 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:00:43.028174 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:43.028174 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 343/666] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 01:00:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:43.028673 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a6a787-7418-4bc7-a0b9-ec770a472c07 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:43.044260 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff6bb5e-c66a-4ea3-ab78-aa913f49f0bf None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:43.062527 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:43.080811 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4930300-ee73-4ba2-9810-ebbd809b3756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:43.081437 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4930300-ee73-4ba2-9810-ebbd809b3756 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:43.081717 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4930300-ee73-4ba2-9810-ebbd809b3756 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:43.081987 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4930300-ee73-4ba2-9810-ebbd809b3756 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:43.093164 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4930300-ee73-4ba2-9810-ebbd809b3756 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:43.094905 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4930300-ee73-4ba2-9810-ebbd809b3756 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:43.109529 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-a4930300-ee73-4ba2-9810-ebbd809b3756 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:43.109529 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:43.109529 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:43.109529 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:43.109529 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:43.109529 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:43.109529 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:43.109529 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:43.109529 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:43.109529 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:43.109529 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:43.109529 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:43.109529 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:00:43.109529 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:00:43.109529 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:00:43.109529 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:00:43.110533 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:43.110533 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:43.110533 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:43.110533 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:43.110533 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:43.110533 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:43.110533 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:43.110533 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:43.110533 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:43.110533 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:43.110533 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:43.110533 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:43.110533 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:43.110533 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:43.110533 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:43.110533 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:43.110533 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:00:43.111409 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:00:43.111409 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:43.111409 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:43.111409 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 324/667] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:00:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:43.111409 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4930300-ee73-4ba2-9810-ebbd809b3756 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:43.148916 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a6a787-7418-4bc7-a0b9-ec770a472c07 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:43.186142 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c31fe-fab5-4717-834e-518014347495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:43.186669 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c31fe-fab5-4717-834e-518014347495 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:43.186953 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c31fe-fab5-4717-834e-518014347495 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:43.187228 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c31fe-fab5-4717-834e-518014347495 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:43.218734 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4930300-ee73-4ba2-9810-ebbd809b3756 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:43.253641 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67524c8-c715-4241-9750-d0b8bde79b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:43.254252 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67524c8-c715-4241-9750-d0b8bde79b22 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:43.254544 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67524c8-c715-4241-9750-d0b8bde79b22 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:43.254817 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67524c8-c715-4241-9750-d0b8bde79b22 None admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:43.255487 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67524c8-c715-4241-9750-d0b8bde79b22 None admin] RBAC: Adding query filter params (name=service) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:43.255792 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67524c8-c715-4241-9750-d0b8bde79b22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:43.257243 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67524c8-c715-4241-9750-d0b8bde79b22 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:43.262937 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:43.270271 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1c31fe-fab5-4717-834e-518014347495 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:43.270271 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1c31fe-fab5-4717-834e-518014347495 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:43.271218 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 325/668] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 8 01:00:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:43.272987 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67524c8-c715-4241-9750-d0b8bde79b22 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:43.298047 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:43.379835 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 344/669] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 01:00:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 01:00:43.380930 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c31fe-fab5-4717-834e-518014347495 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:43.387919 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67524c8-c715-4241-9750-d0b8bde79b22 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:43.422849 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e990239-5333-45a0-9f7e-eb5e56b0bb59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:43.423462 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e990239-5333-45a0-9f7e-eb5e56b0bb59 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:43.423742 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e990239-5333-45a0-9f7e-eb5e56b0bb59 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:43.424015 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e990239-5333-45a0-9f7e-eb5e56b0bb59 None admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:43.424736 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e990239-5333-45a0-9f7e-eb5e56b0bb59 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:43.425043 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e990239-5333-45a0-9f7e-eb5e56b0bb59 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:43.426520 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e990239-5333-45a0-9f7e-eb5e56b0bb59 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:43.438513 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 326/670] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 8 01:00:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:43.439236 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e990239-5333-45a0-9f7e-eb5e56b0bb59 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:43.498146 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c31fe-fab5-4717-834e-518014347495 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:43.502823 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:43.526635 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:43.537252 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcee1be1-e19b-49e2-bc1d-e2c784f50f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:43.537968 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee1be1-e19b-49e2-bc1d-e2c784f50f55 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:43.538246 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee1be1-e19b-49e2-bc1d-e2c784f50f55 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:43.538565 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee1be1-e19b-49e2-bc1d-e2c784f50f55 None admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:43.539148 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcee1be1-e19b-49e2-bc1d-e2c784f50f55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:43.539449 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcee1be1-e19b-49e2-bc1d-e2c784f50f55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:43.541092 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcee1be1-e19b-49e2-bc1d-e2c784f50f55 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:43.546758 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e990239-5333-45a0-9f7e-eb5e56b0bb59 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:43.557540 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 345/671] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:43.557540 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcee1be1-e19b-49e2-bc1d-e2c784f50f55 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:43.583980 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0867aa3d-4f26-4466-b69d-b52fc0ca7e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:43.584645 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0867aa3d-4f26-4466-b69d-b52fc0ca7e1a None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:43.584930 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0867aa3d-4f26-4466-b69d-b52fc0ca7e1a None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:43.585199 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0867aa3d-4f26-4466-b69d-b52fc0ca7e1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:43.596344 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0867aa3d-4f26-4466-b69d-b52fc0ca7e1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:43.598030 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0867aa3d-4f26-4466-b69d-b52fc0ca7e1a None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:43.613419 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-0867aa3d-4f26-4466-b69d-b52fc0ca7e1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:43.613419 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:43.613419 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:43.613419 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:43.613419 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:43.613419 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:43.613419 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:43.613419 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:43.613419 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:43.613419 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:43.613419 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:43.613419 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:43.613419 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:00:43.613419 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:00:43.613419 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:00:43.613419 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:00:43.614419 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:43.614419 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:43.614419 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:43.614419 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:43.614419 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:43.614419 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:43.614419 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:43.614419 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:43.614419 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:43.614419 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:43.614419 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:43.614419 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:43.614419 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:43.614419 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:43.614419 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:43.614419 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:43.614419 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:00:43.615370 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:00:43.615370 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:43.615370 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:43.615370 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 327/672] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:00:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:43.615674 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0867aa3d-4f26-4466-b69d-b52fc0ca7e1a None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:43.675911 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcee1be1-e19b-49e2-bc1d-e2c784f50f55 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:43.711670 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d82dd7-2ff1-48a9-9dbc-fb0f62f04dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:43.712150 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d82dd7-2ff1-48a9-9dbc-fb0f62f04dac None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:43.712459 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d82dd7-2ff1-48a9-9dbc-fb0f62f04dac None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:43.712736 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d82dd7-2ff1-48a9-9dbc-fb0f62f04dac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:43.724786 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d82dd7-2ff1-48a9-9dbc-fb0f62f04dac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:43.726327 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d82dd7-2ff1-48a9-9dbc-fb0f62f04dac None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:43.735886 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0867aa3d-4f26-4466-b69d-b52fc0ca7e1a None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:43.742711 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-11d82dd7-2ff1-48a9-9dbc-fb0f62f04dac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:43.742711 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:43.742711 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:43.742711 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:43.742711 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:43.742711 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:43.742711 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:43.742711 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:43.742711 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:43.742711 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:43.742711 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:43.742711 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:43.742711 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:00:43.742711 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:00:43.742711 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:00:43.742711 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:00:43.743648 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:43.743648 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:43.743648 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:43.743648 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:43.743648 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:43.743648 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:43.743648 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:43.743648 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:43.743648 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:43.743648 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:43.743648 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:43.743648 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:43.743648 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:43.743648 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:43.743648 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:43.743648 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:43.743648 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:00:43.744619 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:00:43.744619 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:43.744619 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:43.744790 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 346/673] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:00:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:43.746635 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d82dd7-2ff1-48a9-9dbc-fb0f62f04dac None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:43.755390 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:43.779536 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36721e3e-73e8-4188-a812-6b6752207b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:43.781466 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36721e3e-73e8-4188-a812-6b6752207b10 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:43.781466 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36721e3e-73e8-4188-a812-6b6752207b10 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:43.781466 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36721e3e-73e8-4188-a812-6b6752207b10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:43.782014 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36721e3e-73e8-4188-a812-6b6752207b10 None admin] RBAC: Adding query filter params (role.id=609d73f56e6a48698397131168abe6df, scope.project.id=78ad2a8fb5264fb4af935e039d28c8de, user.id=d1acfa7a09d34e2a9045c3f3fbea0f52) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:43.782325 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36721e3e-73e8-4188-a812-6b6752207b10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:43.783765 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36721e3e-73e8-4188-a812-6b6752207b10 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:43.790743 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:43.800212 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 328/674] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 8 01:00:43 2022] GET /identity/v3/role_assignments?user.id=d1acfa7a09d34e2a9045c3f3fbea0f52&scope.project.id=78ad2a8fb5264fb4af935e039d28c8de&role.id=609d73f56e6a48698397131168abe6df => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:43.801235 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36721e3e-73e8-4188-a812-6b6752207b10 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:43.863881 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d82dd7-2ff1-48a9-9dbc-fb0f62f04dac None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:43.902893 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd53cf-c4a6-4477-b0b0-0e14cc3e4337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:43.903826 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd53cf-c4a6-4477-b0b0-0e14cc3e4337 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:43.903826 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd53cf-c4a6-4477-b0b0-0e14cc3e4337 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:43.904129 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd53cf-c4a6-4477-b0b0-0e14cc3e4337 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:43.911921 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebd53cf-c4a6-4477-b0b0-0e14cc3e4337 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:43.913552 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebd53cf-c4a6-4477-b0b0-0e14cc3e4337 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:43.926487 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-9ebd53cf-c4a6-4477-b0b0-0e14cc3e4337 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:00:43.926487 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:43.926487 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:43.926487 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:43.926487 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:43.926487 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:43.926487 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:43.926487 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:43.926487 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:43.926487 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:43.926487 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:43.926487 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:43.926487 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 01:00:43.926487 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 01:00:43.926487 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 01:00:43.927459 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 01:00:43.927459 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:43.927459 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:43.927459 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:43.927459 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:43.927459 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:43.927459 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:43.927459 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:43.927459 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:43.927459 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:43.927459 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:43.927459 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:43.927459 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:43.927459 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:43.927459 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:43.927459 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:43.927459 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:43.928372 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 01:00:43.928372 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 01:00:43.928372 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 01:00:43.928372 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 01:00:43.928372 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 01:00:43.928372 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 01:00:43.928372 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:00:43.928372 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:43.928372 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 347/675] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 01:00:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:43.929549 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd53cf-c4a6-4477-b0b0-0e14cc3e4337 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:43.961154 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36721e3e-73e8-4188-a812-6b6752207b10 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:44.012013 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31db6db-8ebd-4661-9371-e9a6d65abe27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:44.012658 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31db6db-8ebd-4661-9371-e9a6d65abe27 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:44.012937 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31db6db-8ebd-4661-9371-e9a6d65abe27 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:44.013211 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31db6db-8ebd-4661-9371-e9a6d65abe27 None admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:44.013855 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31db6db-8ebd-4661-9371-e9a6d65abe27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:44.014158 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31db6db-8ebd-4661-9371-e9a6d65abe27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:44.015546 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31db6db-8ebd-4661-9371-e9a6d65abe27 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:44.030524 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 329/676] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:44.035233 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31db6db-8ebd-4661-9371-e9a6d65abe27 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:44.053839 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:44.059265 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd53cf-c4a6-4477-b0b0-0e14cc3e4337 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:44.080061 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:44.110541 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4744703-8037-4ff1-9e17-39d63bca0629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:44.111157 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4744703-8037-4ff1-9e17-39d63bca0629 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:44.111447 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4744703-8037-4ff1-9e17-39d63bca0629 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:44.111715 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4744703-8037-4ff1-9e17-39d63bca0629 None admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:44.112416 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4744703-8037-4ff1-9e17-39d63bca0629 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:44.112723 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4744703-8037-4ff1-9e17-39d63bca0629 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:44.114104 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4744703-8037-4ff1-9e17-39d63bca0629 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:44.127111 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 348/677] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:44.127956 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4744703-8037-4ff1-9e17-39d63bca0629 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:44.164402 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31db6db-8ebd-4661-9371-e9a6d65abe27 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:44.208514 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a017682-6d2c-47d3-8aa0-e1e807d844d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:44.209539 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a017682-6d2c-47d3-8aa0-e1e807d844d7 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:44.209828 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a017682-6d2c-47d3-8aa0-e1e807d844d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:44.210380 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a017682-6d2c-47d3-8aa0-e1e807d844d7 None admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:44.211095 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a017682-6d2c-47d3-8aa0-e1e807d844d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:44.211408 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a017682-6d2c-47d3-8aa0-e1e807d844d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:44.213147 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a017682-6d2c-47d3-8aa0-e1e807d844d7 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:44.229095 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 330/678] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 8 01:00:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:44.230872 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a017682-6d2c-47d3-8aa0-e1e807d844d7 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:44.257516 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4744703-8037-4ff1-9e17-39d63bca0629 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:44.303457 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cbad91-70b0-476b-8c6b-5ed0d6532fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:44.304330 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cbad91-70b0-476b-8c6b-5ed0d6532fbe None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:44.304622 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cbad91-70b0-476b-8c6b-5ed0d6532fbe None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:44.304924 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cbad91-70b0-476b-8c6b-5ed0d6532fbe None admin] PATH_INFO: `/v3/users/swift` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:44.315421 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cbad91-70b0-476b-8c6b-5ed0d6532fbe None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:44.316928 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cbad91-70b0-476b-8c6b-5ed0d6532fbe None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:44.329784 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-60cbad91-70b0-476b-8c6b-5ed0d6532fbe None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 08 01:00:44.329784 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:44.329784 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:44.329784 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:44.329784 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:44.329784 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:44.329784 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:44.329784 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:44.329784 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:44.329784 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:44.329784 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:44.329784 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:44.329784 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 01:00:44.329784 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 01:00:44.329784 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 01:00:44.329784 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 01:00:44.331000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:44.331000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:44.331000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 01:00:44.331000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:00:44.331000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 01:00:44.331000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:00:44.331000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:44.331000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:44.331000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:44.331000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:44.331000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:44.331000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:44.331000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:44.331000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:44.331000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:44.331000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:44.331000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:44.331000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:44.331958 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:44.331958 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:44.331958 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 01:00:44.331958 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 01:00:44.331958 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 01:00:44.331958 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 01:00:44.331958 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 01:00:44.331958 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 01:00:44.331958 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 08 01:00:44.331958 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:44.335783 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 349/679] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 8 01:00:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:44.336830 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cbad91-70b0-476b-8c6b-5ed0d6532fbe None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:44.356668 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:44.359982 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:44.369598 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a017682-6d2c-47d3-8aa0-e1e807d844d7 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:44.416249 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd70756f-a519-40d9-9d65-9c57faf0235d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:44.417204 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd70756f-a519-40d9-9d65-9c57faf0235d None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:44.417489 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd70756f-a519-40d9-9d65-9c57faf0235d None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:44.417833 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd70756f-a519-40d9-9d65-9c57faf0235d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:44.433204 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd70756f-a519-40d9-9d65-9c57faf0235d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:44.435203 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd70756f-a519-40d9-9d65-9c57faf0235d None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:44.447760 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cbad91-70b0-476b-8c6b-5ed0d6532fbe None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:44.453283 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-cd70756f-a519-40d9-9d65-9c57faf0235d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:00:44.453283 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:44.453283 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:44.453283 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:44.453283 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:44.453283 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:44.453283 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:44.453283 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:44.453283 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:44.453283 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:44.453283 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:44.453283 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:44.453283 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 01:00:44.453283 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 01:00:44.453283 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 01:00:44.454182 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 01:00:44.454182 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:44.454182 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:44.454182 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:44.454182 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:44.454182 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:44.454182 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:44.454182 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:44.454182 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:44.454182 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:44.454182 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:44.454182 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:44.454182 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:44.454182 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:44.454182 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:44.454182 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:44.454182 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:44.455093 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 01:00:44.455093 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 01:00:44.455093 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 01:00:44.455093 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 01:00:44.455093 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 01:00:44.455093 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 01:00:44.455093 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:00:44.455093 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:44.462663 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 331/680] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 01:00:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:44.462663 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd70756f-a519-40d9-9d65-9c57faf0235d None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:44.490604 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfedb2ec-c49a-4da9-a484-0bc1af927f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:44.491199 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfedb2ec-c49a-4da9-a484-0bc1af927f70 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:44.491484 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfedb2ec-c49a-4da9-a484-0bc1af927f70 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:44.491759 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfedb2ec-c49a-4da9-a484-0bc1af927f70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:44.492522 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfedb2ec-c49a-4da9-a484-0bc1af927f70 None admin] RBAC: Adding query filter params (role.id=609d73f56e6a48698397131168abe6df, scope.project.id=78ad2a8fb5264fb4af935e039d28c8de, user.id=a307e9150ead4877b970e09f36553968) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:44.492859 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfedb2ec-c49a-4da9-a484-0bc1af927f70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:44.494265 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfedb2ec-c49a-4da9-a484-0bc1af927f70 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:44.501484 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 350/681] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 8 01:00:44 2022] GET /identity/v3/role_assignments?user.id=a307e9150ead4877b970e09f36553968&scope.project.id=78ad2a8fb5264fb4af935e039d28c8de&role.id=609d73f56e6a48698397131168abe6df => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:44.502298 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfedb2ec-c49a-4da9-a484-0bc1af927f70 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:44.602097 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd70756f-a519-40d9-9d65-9c57faf0235d None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:44.613770 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:44.650590 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ea15f-f6c6-4c6b-a571-5a41121c740b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:44.651631 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ea15f-f6c6-4c6b-a571-5a41121c740b None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:44.652788 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ea15f-f6c6-4c6b-a571-5a41121c740b None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:44.653342 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ea15f-f6c6-4c6b-a571-5a41121c740b None admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:44.654322 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7ea15f-f6c6-4c6b-a571-5a41121c740b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:44.654440 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfedb2ec-c49a-4da9-a484-0bc1af927f70 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:44.655012 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7ea15f-f6c6-4c6b-a571-5a41121c740b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:44.656782 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7ea15f-f6c6-4c6b-a571-5a41121c740b None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:44.683781 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 332/682] 10.222.0.42 () {56 vars in 1135 bytes} [Thu Sep 8 01:00:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:44.688164 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ea15f-f6c6-4c6b-a571-5a41121c740b None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:44.707881 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d27f9-5fc6-4887-9c8f-17fbaa3c7eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:44.708907 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d27f9-5fc6-4887-9c8f-17fbaa3c7eb4 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:44.709459 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d27f9-5fc6-4887-9c8f-17fbaa3c7eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:44.710472 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d27f9-5fc6-4887-9c8f-17fbaa3c7eb4 None admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:44.712398 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007d27f9-5fc6-4887-9c8f-17fbaa3c7eb4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:44.713700 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007d27f9-5fc6-4887-9c8f-17fbaa3c7eb4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:44.716887 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007d27f9-5fc6-4887-9c8f-17fbaa3c7eb4 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:44.721520 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:44.743992 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 351/683] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 8 01:00:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:44.755885 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d27f9-5fc6-4887-9c8f-17fbaa3c7eb4 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:44.863683 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ea15f-f6c6-4c6b-a571-5a41121c740b None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:44.903797 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d27f9-5fc6-4887-9c8f-17fbaa3c7eb4 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:44.907748 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91371d81-fde3-4951-a24b-46635c595061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:44.908592 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91371d81-fde3-4951-a24b-46635c595061 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:44.909074 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91371d81-fde3-4951-a24b-46635c595061 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:44.909492 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91371d81-fde3-4951-a24b-46635c595061 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:44.941613 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7006f19-941d-4906-a0e3-5e3711eb325a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:44.942562 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7006f19-941d-4906-a0e3-5e3711eb325a None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:44.942786 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7006f19-941d-4906-a0e3-5e3711eb325a None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:44.943158 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7006f19-941d-4906-a0e3-5e3711eb325a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:44.943992 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7006f19-941d-4906-a0e3-5e3711eb325a None admin] RBAC: Adding query filter params (role.id=609d73f56e6a48698397131168abe6df, scope.project.id=78ad2a8fb5264fb4af935e039d28c8de, user.id=5dbbcc3dbcc745458e97b31c6781ce70) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:44.944460 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7006f19-941d-4906-a0e3-5e3711eb325a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:44.946376 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7006f19-941d-4906-a0e3-5e3711eb325a None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:44.953369 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 352/684] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 8 01:00:44 2022] GET /identity/v3/role_assignments?user.id=5dbbcc3dbcc745458e97b31c6781ce70&scope.project.id=78ad2a8fb5264fb4af935e039d28c8de&role.id=609d73f56e6a48698397131168abe6df => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:00:44.954790 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2edb96-5327-4201-9ce5-d925053f0f59 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:44.955883 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2edb96-5327-4201-9ce5-d925053f0f59 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:44.955883 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2edb96-5327-4201-9ce5-d925053f0f59 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:44.957230 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 353/685] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:00:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:00:44.968565 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44142b-7f3a-48d9-9ef5-f42a2d4538ed None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:44.968565 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44142b-7f3a-48d9-9ef5-f42a2d4538ed None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:44.968964 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44142b-7f3a-48d9-9ef5-f42a2d4538ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:44.993710 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:44.996801 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91371d81-fde3-4951-a24b-46635c595061 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:44.998613 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91371d81-fde3-4951-a24b-46635c595061 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:45.002736 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-af44142b-7f3a-48d9-9ef5-f42a2d4538ed None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:00:45.004216 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-af44142b-7f3a-48d9-9ef5-f42a2d4538ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:00:45.018285 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:45.129868 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 333/686] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 01:00:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 08 01:00:45.138666 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91371d81-fde3-4951-a24b-46635c595061 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:45.156370 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 354/687] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:00:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 01:00:45.261822 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91371d81-fde3-4951-a24b-46635c595061 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:45.278032 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:45.299973 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a27282-6bda-4aaf-943c-c0a25b49de67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:45.300633 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a27282-6bda-4aaf-943c-c0a25b49de67 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:45.300913 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a27282-6bda-4aaf-943c-c0a25b49de67 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:45.301185 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a27282-6bda-4aaf-943c-c0a25b49de67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:45.312048 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a27282-6bda-4aaf-943c-c0a25b49de67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:45.313538 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a27282-6bda-4aaf-943c-c0a25b49de67 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:45.328925 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-36a27282-6bda-4aaf-943c-c0a25b49de67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:45.328925 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:45.328925 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:45.328925 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:45.328925 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:45.328925 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:45.328925 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:45.328925 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:45.328925 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:45.328925 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:45.328925 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:45.328925 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:45.328925 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:00:45.328925 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:00:45.328925 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:00:45.328925 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:00:45.329901 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:45.329901 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:45.329901 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:45.329901 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:45.329901 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:45.329901 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:45.329901 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:45.329901 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:45.329901 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:45.329901 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:45.329901 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:45.329901 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:45.329901 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:45.329901 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:45.329901 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:45.329901 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:45.329901 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:00:45.331231 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:00:45.331231 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:45.331231 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:45.331231 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 334/688] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:00:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:45.338481 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af44142b-7f3a-48d9-9ef5-f42a2d4538ed None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:45.447133 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af44142b-7f3a-48d9-9ef5-f42a2d4538ed None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:45.485891 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc94bbe1-2e64-45ab-bd1e-975ce4076b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:45.486869 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc94bbe1-2e64-45ab-bd1e-975ce4076b9f None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:45.487353 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc94bbe1-2e64-45ab-bd1e-975ce4076b9f None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:45.487866 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc94bbe1-2e64-45ab-bd1e-975ce4076b9f None admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:45.488767 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc94bbe1-2e64-45ab-bd1e-975ce4076b9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:45.489348 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc94bbe1-2e64-45ab-bd1e-975ce4076b9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:45.490925 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc94bbe1-2e64-45ab-bd1e-975ce4076b9f None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:45.492695 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a27282-6bda-4aaf-943c-c0a25b49de67 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:45.493921 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:45.506636 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 355/689] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:00:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:45.520264 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc94bbe1-2e64-45ab-bd1e-975ce4076b9f None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:45.664579 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc94bbe1-2e64-45ab-bd1e-975ce4076b9f None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:45.704031 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a181234-53e1-4bea-9bb4-740c75884c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:45.706034 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a181234-53e1-4bea-9bb4-740c75884c13 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:45.706752 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a181234-53e1-4bea-9bb4-740c75884c13 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:45.707843 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a181234-53e1-4bea-9bb4-740c75884c13 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:45.718740 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a181234-53e1-4bea-9bb4-740c75884c13 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:45.720633 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a181234-53e1-4bea-9bb4-740c75884c13 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:45.723234 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a27282-6bda-4aaf-943c-c0a25b49de67 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:45.737779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-6a181234-53e1-4bea-9bb4-740c75884c13 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:00:45.737779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:45.737779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:45.737779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:45.737779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:45.737779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:45.737779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:45.737779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:45.737779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:45.737779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:45.737779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:45.737779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:45.737779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 01:00:45.737779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 01:00:45.737779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 01:00:45.738814 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 01:00:45.738814 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:45.738814 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:45.738814 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:45.738814 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:45.738814 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:45.738814 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:45.738814 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:45.738814 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:45.738814 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:45.738814 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:45.738814 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:45.738814 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:45.738814 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:45.738814 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:45.738814 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:45.738814 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:45.739711 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 01:00:45.739711 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 01:00:45.739711 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 01:00:45.739711 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 01:00:45.739711 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 01:00:45.739711 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 01:00:45.739711 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:00:45.739711 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:45.742452 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 356/690] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 01:00:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:45.749962 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a181234-53e1-4bea-9bb4-740c75884c13 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:45.799013 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599fa20e-8fa1-4e43-9689-0f897aa59a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:45.800040 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599fa20e-8fa1-4e43-9689-0f897aa59a9b None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:45.800547 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599fa20e-8fa1-4e43-9689-0f897aa59a9b None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:45.802255 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599fa20e-8fa1-4e43-9689-0f897aa59a9b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:45.829074 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599fa20e-8fa1-4e43-9689-0f897aa59a9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:00:45.834443 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-599fa20e-8fa1-4e43-9689-0f897aa59a9b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 01:00:45.834443 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:45.834443 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:45.834443 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:45.834443 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:45.834443 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:45.834443 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:45.834443 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:45.834443 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:45.834443 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:45.834443 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:45.834443 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:45.834443 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:00:45.834443 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:00:45.834443 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:00:45.834443 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:00:45.835374 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:00:45.835374 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:00:45.835374 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:00:45.835374 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:00:45.835374 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:45.835374 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:45.835374 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:45.835374 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:45.835374 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:45.835374 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:45.835374 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:45.835374 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:45.835374 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:45.835374 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:45.835374 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:45.835374 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:45.836235 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:45.836235 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:45.836235 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:45.836235 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:45.836235 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:00:45.836235 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:00:45.836235 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:00:45.836235 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:00:45.836235 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:00:45.836235 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:00:45.836235 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 01:00:45.836235 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:45.836235 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 335/691] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 8 01:00:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:45.845185 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599fa20e-8fa1-4e43-9689-0f897aa59a9b None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:45.847962 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:45.900103 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a181234-53e1-4bea-9bb4-740c75884c13 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:45.901737 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:45.941318 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b93ca4-5473-4959-818a-3d7344f4563f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:45.942317 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b93ca4-5473-4959-818a-3d7344f4563f None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:45.942905 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b93ca4-5473-4959-818a-3d7344f4563f None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:45.943428 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b93ca4-5473-4959-818a-3d7344f4563f None admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:45.944346 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b93ca4-5473-4959-818a-3d7344f4563f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:45.944870 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b93ca4-5473-4959-818a-3d7344f4563f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:45.946604 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b93ca4-5473-4959-818a-3d7344f4563f None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:45.960918 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 357/692] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 8 01:00:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:00:45.968018 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b93ca4-5473-4959-818a-3d7344f4563f None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:46.041242 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599fa20e-8fa1-4e43-9689-0f897aa59a9b None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:46.079994 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b93ca4-5473-4959-818a-3d7344f4563f None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:46.099248 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:46.121484 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf4600-dff3-4d1e-923e-40648fa83f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:46.121882 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbf5b3f-19db-4e33-8315-0abdfae0680d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:46.122538 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf5b3f-19db-4e33-8315-0abdfae0680d None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:46.122821 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf5b3f-19db-4e33-8315-0abdfae0680d None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:46.123097 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf5b3f-19db-4e33-8315-0abdfae0680d None admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:46.123659 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf4600-dff3-4d1e-923e-40648fa83f06 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:46.123837 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbf5b3f-19db-4e33-8315-0abdfae0680d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:46.124100 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbf5b3f-19db-4e33-8315-0abdfae0680d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:46.124525 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf4600-dff3-4d1e-923e-40648fa83f06 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:46.125049 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf4600-dff3-4d1e-923e-40648fa83f06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:46.125556 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbf5b3f-19db-4e33-8315-0abdfae0680d None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:46.126280 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf4600-dff3-4d1e-923e-40648fa83f06 None admin] RBAC: Adding query filter params (role.id=609d73f56e6a48698397131168abe6df, scope.project.id=78ad2a8fb5264fb4af935e039d28c8de, user.id=8add26872fe6479aa9d7de155a17195a) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:46.126797 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf4600-dff3-4d1e-923e-40648fa83f06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:46.128631 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf4600-dff3-4d1e-923e-40648fa83f06 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:46.137382 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 358/693] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 8 01:00:45 2022] GET /identity/v3/role_assignments?user.id=8add26872fe6479aa9d7de155a17195a&scope.project.id=78ad2a8fb5264fb4af935e039d28c8de&role.id=609d73f56e6a48698397131168abe6df => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:46.147212 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 336/694] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 01:00:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:46.154269 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf4600-dff3-4d1e-923e-40648fa83f06 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:46.273249 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf4600-dff3-4d1e-923e-40648fa83f06 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:46.312351 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a5c703-bce8-43b3-a873-731e3cc167c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:46.313408 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a5c703-bce8-43b3-a873-731e3cc167c6 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:46.313927 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a5c703-bce8-43b3-a873-731e3cc167c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:46.314424 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a5c703-bce8-43b3-a873-731e3cc167c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:46.328310 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a5c703-bce8-43b3-a873-731e3cc167c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:46.330220 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a5c703-bce8-43b3-a873-731e3cc167c6 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:46.333833 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518d7a8a-0d30-437b-a2d1-1c245e199b4e None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:46.334373 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518d7a8a-0d30-437b-a2d1-1c245e199b4e None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:46.334373 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518d7a8a-0d30-437b-a2d1-1c245e199b4e None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:46.335912 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 337/695] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:00:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:00:46.342052 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce5a2ae-b98b-4f73-a6a3-644ab2fceedb None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:46.342360 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce5a2ae-b98b-4f73-a6a3-644ab2fceedb None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:46.342607 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce5a2ae-b98b-4f73-a6a3-644ab2fceedb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:46.349659 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-c0a5c703-bce8-43b3-a873-731e3cc167c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:46.349659 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:46.349659 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:46.349659 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:46.349659 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:46.349659 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:46.349659 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:46.349659 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:46.349659 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:46.349659 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:46.349659 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:46.349659 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:46.349659 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:00:46.349659 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:00:46.349659 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:00:46.349659 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:00:46.350987 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:46.350987 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:46.350987 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:46.350987 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:46.350987 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:46.350987 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:46.350987 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:46.350987 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:46.350987 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:46.350987 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:46.350987 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:46.350987 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:46.350987 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:46.350987 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:46.350987 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:46.350987 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:46.350987 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:00:46.351879 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:00:46.351879 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:46.351879 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:46.354353 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 359/696] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:00:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:46.375888 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-fce5a2ae-b98b-4f73-a6a3-644ab2fceedb None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:00:46.376572 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a5c703-bce8-43b3-a873-731e3cc167c6 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:46.377165 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-fce5a2ae-b98b-4f73-a6a3-644ab2fceedb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:00:46.413009 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:46.469235 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 338/697] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:00:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 01:00:46.494670 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a5c703-bce8-43b3-a873-731e3cc167c6 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:46.532207 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c43715-a375-4e1b-ab25-5e747fd87da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:46.533129 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c43715-a375-4e1b-ab25-5e747fd87da2 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:46.533657 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c43715-a375-4e1b-ab25-5e747fd87da2 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:46.534129 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c43715-a375-4e1b-ab25-5e747fd87da2 None admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:46.534971 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c43715-a375-4e1b-ab25-5e747fd87da2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:46.535646 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c43715-a375-4e1b-ab25-5e747fd87da2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:46.537368 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c43715-a375-4e1b-ab25-5e747fd87da2 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:46.550524 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 360/698] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:00:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:46.555962 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce5a2ae-b98b-4f73-a6a3-644ab2fceedb None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:46.577515 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:46.667427 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce5a2ae-b98b-4f73-a6a3-644ab2fceedb None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:46.703552 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd2507-85cb-4718-87e7-5add76938cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:46.704204 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd2507-85cb-4718-87e7-5add76938cbb None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:46.704490 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd2507-85cb-4718-87e7-5add76938cbb None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:46.704763 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd2507-85cb-4718-87e7-5add76938cbb None admin] PATH_INFO: `/v3/users/nova` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:46.712294 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bd2507-85cb-4718-87e7-5add76938cbb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:46.713749 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bd2507-85cb-4718-87e7-5add76938cbb None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:46.726143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-24bd2507-85cb-4718-87e7-5add76938cbb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 01:00:46.726143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:46.726143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:46.726143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:46.726143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:46.726143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:46.726143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:46.726143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:46.726143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:46.726143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:46.726143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:46.726143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:46.726143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 01:00:46.726143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 01:00:46.726143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 01:00:46.726143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 01:00:46.727127 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:46.727127 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:46.727127 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 01:00:46.727127 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:00:46.727127 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 01:00:46.727127 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:00:46.727127 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:46.727127 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:46.727127 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:46.727127 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:46.727127 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:46.727127 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:46.727127 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:46.727127 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:46.727127 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:46.727127 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:46.727127 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:46.727127 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:46.728051 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:46.728051 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:46.728051 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 01:00:46.728051 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 01:00:46.728051 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 01:00:46.728051 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 01:00:46.728051 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 01:00:46.728051 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 01:00:46.728051 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 01:00:46.728051 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:46.728638 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 339/699] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Sep 8 01:00:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:46.733623 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c43715-a375-4e1b-ab25-5e747fd87da2 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:46.775237 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd2507-85cb-4718-87e7-5add76938cbb None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:46.849452 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:46.860343 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c43715-a375-4e1b-ab25-5e747fd87da2 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:46.868440 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:46.899365 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa96f725-2e72-4d48-af2c-ae1daa6e3686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:46.900151 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96f725-2e72-4d48-af2c-ae1daa6e3686 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:46.900535 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96f725-2e72-4d48-af2c-ae1daa6e3686 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:46.900904 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96f725-2e72-4d48-af2c-ae1daa6e3686 None admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:46.901737 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa96f725-2e72-4d48-af2c-ae1daa6e3686 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:46.902778 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa96f725-2e72-4d48-af2c-ae1daa6e3686 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:46.905380 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa96f725-2e72-4d48-af2c-ae1daa6e3686 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:46.912442 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd2507-85cb-4718-87e7-5add76938cbb None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:46.926656 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 361/700] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Sep 8 01:00:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:46.938057 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa96f725-2e72-4d48-af2c-ae1daa6e3686 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:46.955683 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4398d9-96e8-44b6-93e4-b11d0236af75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:46.956361 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4398d9-96e8-44b6-93e4-b11d0236af75 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:46.956653 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4398d9-96e8-44b6-93e4-b11d0236af75 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:46.956917 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4398d9-96e8-44b6-93e4-b11d0236af75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:46.973777 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4398d9-96e8-44b6-93e4-b11d0236af75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:46.975490 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4398d9-96e8-44b6-93e4-b11d0236af75 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:46.999601 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-7b4398d9-96e8-44b6-93e4-b11d0236af75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:46.999601 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:46.999601 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:46.999601 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:46.999601 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:46.999601 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:46.999601 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:46.999601 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:46.999601 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:46.999601 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:46.999601 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:46.999601 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:46.999601 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:00:46.999601 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:00:46.999601 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:00:46.999601 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:00:47.000821 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:47.000821 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:47.000821 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:47.000821 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:47.000821 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:47.000821 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:47.000821 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:47.000821 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:47.000821 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:47.000821 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:47.000821 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:47.000821 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:47.000821 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:47.000821 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:47.000821 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:47.000821 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:47.000821 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:00:47.001868 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:00:47.001868 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:47.001868 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:47.001868 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 340/701] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:00:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 01:00:47.025583 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4398d9-96e8-44b6-93e4-b11d0236af75 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:47.139824 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa96f725-2e72-4d48-af2c-ae1daa6e3686 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:47.195548 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843f1485-e076-4235-935e-f7635156f4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:47.196549 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843f1485-e076-4235-935e-f7635156f4cd None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:47.196981 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843f1485-e076-4235-935e-f7635156f4cd None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:47.197400 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843f1485-e076-4235-935e-f7635156f4cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:47.199056 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4398d9-96e8-44b6-93e4-b11d0236af75 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:47.235372 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:47.239683 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:47.251026 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf3f8b1-06ce-4701-b01c-dd1ff42f7800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:47.252116 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3f8b1-06ce-4701-b01c-dd1ff42f7800 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:47.252768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3f8b1-06ce-4701-b01c-dd1ff42f7800 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:47.253205 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3f8b1-06ce-4701-b01c-dd1ff42f7800 None admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:47.253987 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf3f8b1-06ce-4701-b01c-dd1ff42f7800 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:47.254581 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf3f8b1-06ce-4701-b01c-dd1ff42f7800 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:47.257608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf3f8b1-06ce-4701-b01c-dd1ff42f7800 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:47.285832 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 341/702] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:00:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:47.291501 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf3f8b1-06ce-4701-b01c-dd1ff42f7800 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:47.334828 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843f1485-e076-4235-935e-f7635156f4cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:47.337819 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843f1485-e076-4235-935e-f7635156f4cd None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:47.468473 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf3f8b1-06ce-4701-b01c-dd1ff42f7800 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:47.516650 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 362/703] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 01:00:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 08 01:00:47.516650 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ea33d-6696-4d73-8df5-6deaf454494a None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:47.516650 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ea33d-6696-4d73-8df5-6deaf454494a None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:47.516650 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ea33d-6696-4d73-8df5-6deaf454494a None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:47.516650 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 363/704] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:00:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:00:47.520150 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1ea33d-6696-4d73-8df5-6deaf454494a None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:47.525199 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163a1735-d2f4-4b0a-b04c-171588f29e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:47.525824 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a1735-d2f4-4b0a-b04c-171588f29e6f None admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:47.526109 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a1735-d2f4-4b0a-b04c-171588f29e6f None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:47.526380 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a1735-d2f4-4b0a-b04c-171588f29e6f None admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:47.527113 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163a1735-d2f4-4b0a-b04c-171588f29e6f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:47.528833 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163a1735-d2f4-4b0a-b04c-171588f29e6f None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:47.557887 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 342/705] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 8 01:00:47 2022] POST /identity/v3/users => generated 254 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:00:47.559906 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3effbad0-6a81-407d-8ae8-a224374d5f20 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:47.560218 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3effbad0-6a81-407d-8ae8-a224374d5f20 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:47.560504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3effbad0-6a81-407d-8ae8-a224374d5f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:47.583024 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:47.584976 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-3effbad0-6a81-407d-8ae8-a224374d5f20 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:00:47.586233 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-3effbad0-6a81-407d-8ae8-a224374d5f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:00:47.658519 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1ea33d-6696-4d73-8df5-6deaf454494a None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:47.671049 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:47.731062 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 343/706] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:00:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 01:00:47.732732 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4480790b-f426-4777-a594-abe70a8a65f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:47.733452 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4480790b-f426-4777-a594-abe70a8a65f3 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:47.733866 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4480790b-f426-4777-a594-abe70a8a65f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:47.734139 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4480790b-f426-4777-a594-abe70a8a65f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:47.759961 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4480790b-f426-4777-a594-abe70a8a65f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:47.762313 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4480790b-f426-4777-a594-abe70a8a65f3 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:47.787732 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-4480790b-f426-4777-a594-abe70a8a65f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:47.787732 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:47.787732 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:47.787732 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:47.787732 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:47.787732 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:47.787732 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:47.787732 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:47.787732 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:47.787732 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:47.787732 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:47.787732 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:47.787732 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:00:47.787732 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:00:47.787732 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:00:47.787732 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:00:47.788812 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:47.788812 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:47.788812 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:47.788812 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:47.788812 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:47.788812 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:47.788812 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:47.788812 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:47.788812 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:47.788812 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:47.788812 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:47.788812 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:47.788812 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:47.788812 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:47.788812 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:47.788812 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:47.788812 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:00:47.789823 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:00:47.789823 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:47.789823 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:47.789823 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 364/707] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:00:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:47.795640 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3effbad0-6a81-407d-8ae8-a224374d5f20 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:47.845116 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e3d9ef-f768-4f84-94f2-9bc4ee77a1cb None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:47.845595 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e3d9ef-f768-4f84-94f2-9bc4ee77a1cb None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:47.845962 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e3d9ef-f768-4f84-94f2-9bc4ee77a1cb None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:47.847679 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 365/708] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:00:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 01:00:47.856313 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff952018-f56e-4140-9063-edb47555b720 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:47.856979 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff952018-f56e-4140-9063-edb47555b720 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:47.857351 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff952018-f56e-4140-9063-edb47555b720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:47.880370 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-ff952018-f56e-4140-9063-edb47555b720 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:00:47.881559 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-ff952018-f56e-4140-9063-edb47555b720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:00:47.941278 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3effbad0-6a81-407d-8ae8-a224374d5f20 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:47.979136 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 366/709] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:00:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 01:00:47.992934 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38798d1e-c15d-4edb-8340-d731d6d0d613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:47.993802 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38798d1e-c15d-4edb-8340-d731d6d0d613 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:47.994257 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38798d1e-c15d-4edb-8340-d731d6d0d613 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:47.994680 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38798d1e-c15d-4edb-8340-d731d6d0d613 None admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:47.995497 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38798d1e-c15d-4edb-8340-d731d6d0d613 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:47.995973 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38798d1e-c15d-4edb-8340-d731d6d0d613 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:47.997569 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38798d1e-c15d-4edb-8340-d731d6d0d613 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:48.013110 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 344/710] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:00:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:48.020845 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff952018-f56e-4140-9063-edb47555b720 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:48.033840 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38798d1e-c15d-4edb-8340-d731d6d0d613 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:48.061498 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:48.106632 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:48.178537 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff952018-f56e-4140-9063-edb47555b720 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:48.225012 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3d61e-2ac9-4047-9837-126dc0d67d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:48.225651 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3d61e-2ac9-4047-9837-126dc0d67d8c None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:48.225928 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3d61e-2ac9-4047-9837-126dc0d67d8c None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:48.226198 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3d61e-2ac9-4047-9837-126dc0d67d8c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:48.237123 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf3d61e-2ac9-4047-9837-126dc0d67d8c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:48.238563 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf3d61e-2ac9-4047-9837-126dc0d67d8c None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:48.253661 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38798d1e-c15d-4edb-8340-d731d6d0d613 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:48.267171 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-cdf3d61e-2ac9-4047-9837-126dc0d67d8c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:00:48.267171 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:48.267171 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:48.267171 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:48.267171 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:48.267171 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:48.267171 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:48.267171 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:48.267171 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:48.267171 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:48.267171 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:48.267171 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:48.267171 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 01:00:48.267171 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 01:00:48.267171 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 01:00:48.268080 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 01:00:48.268080 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:48.268080 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:48.268080 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:48.268080 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:48.268080 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:48.268080 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:48.268080 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:48.268080 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:48.268080 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:48.268080 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:48.268080 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:48.268080 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:48.268080 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:48.268080 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:48.268080 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:48.268080 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:48.268985 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 01:00:48.268985 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 01:00:48.268985 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 01:00:48.268985 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 01:00:48.268985 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 01:00:48.268985 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 01:00:48.268985 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:00:48.268985 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:48.268985 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 367/711] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 01:00:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:48.281073 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3d61e-2ac9-4047-9837-126dc0d67d8c None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:48.298303 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142cf1d6-ea18-4b88-9520-3b4c800551ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:48.298945 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142cf1d6-ea18-4b88-9520-3b4c800551ec None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:48.299223 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142cf1d6-ea18-4b88-9520-3b4c800551ec None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:48.299496 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142cf1d6-ea18-4b88-9520-3b4c800551ec None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:48.300077 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142cf1d6-ea18-4b88-9520-3b4c800551ec None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:48.301500 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142cf1d6-ea18-4b88-9520-3b4c800551ec None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:48.314885 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-142cf1d6-ea18-4b88-9520-3b4c800551ec None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 01:00:48.314885 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:48.314885 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 01:00:48.314885 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 01:00:48.314885 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 01:00:48.314885 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 01:00:48.314885 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 01:00:48.314885 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 01:00:48.314885 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 01:00:48.314885 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:48.314885 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 01:00:48.314885 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:48.314885 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:48.314885 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:48.314885 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:48.314885 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:48.314885 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:48.315952 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:48.315952 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:48.315952 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:48.315952 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:48.315952 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:48.315952 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:48.315952 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 01:00:48.315952 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 01:00:48.315952 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 01:00:48.315952 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 01:00:48.315952 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:48.315952 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:48.315952 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:48.315952 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:48.315952 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:48.315952 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:48.316822 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:48.316822 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:48.316822 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:48.316822 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:48.316822 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:48.316822 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:48.316822 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:48.316822 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:48.316822 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:48.316822 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:48.316822 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 01:00:48.316822 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 01:00:48.316822 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 01:00:48.316822 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:48.316822 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 345/712] 10.222.0.42 () {56 vars in 1129 bytes} [Thu Sep 8 01:00:48 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:48.319086 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142cf1d6-ea18-4b88-9520-3b4c800551ec None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:48.400359 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3d61e-2ac9-4047-9837-126dc0d67d8c None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:48.404498 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:48.439962 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:48.443117 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142cf1d6-ea18-4b88-9520-3b4c800551ec None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:48.443763 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a414c1db-731d-4626-ae59-0a0c4358b25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:48.444592 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a414c1db-731d-4626-ae59-0a0c4358b25c None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:48.444873 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a414c1db-731d-4626-ae59-0a0c4358b25c None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:48.445132 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a414c1db-731d-4626-ae59-0a0c4358b25c None admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:48.445837 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a414c1db-731d-4626-ae59-0a0c4358b25c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:48.446140 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a414c1db-731d-4626-ae59-0a0c4358b25c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:48.447579 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a414c1db-731d-4626-ae59-0a0c4358b25c None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:48.464280 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 368/713] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 8 01:00:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:48.464280 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a414c1db-731d-4626-ae59-0a0c4358b25c None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:48.496725 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a82847-a8d0-4221-a6c2-490925632151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:48.497729 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a82847-a8d0-4221-a6c2-490925632151 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:48.498175 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a82847-a8d0-4221-a6c2-490925632151 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:48.498515 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a82847-a8d0-4221-a6c2-490925632151 None admin] PATH_INFO: `/v3/services/network` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:48.499191 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a82847-a8d0-4221-a6c2-490925632151 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:48.501642 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a82847-a8d0-4221-a6c2-490925632151 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:48.520403 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-c2a82847-a8d0-4221-a6c2-490925632151 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 01:00:48.520403 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:48.520403 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 01:00:48.520403 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 01:00:48.520403 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 01:00:48.520403 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 01:00:48.520403 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 01:00:48.520403 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 01:00:48.520403 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 01:00:48.520403 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:48.520403 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 01:00:48.520403 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:48.520403 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:48.520403 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:48.520403 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:48.520403 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:48.520403 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:48.521417 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:48.521417 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:48.521417 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:48.521417 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:48.521417 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:48.521417 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:48.521417 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 01:00:48.521417 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 01:00:48.521417 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 01:00:48.521417 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 01:00:48.521417 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:48.521417 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:48.521417 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:48.521417 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:48.521417 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:48.521417 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:48.522282 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:48.522282 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:48.522282 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:48.522282 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:48.522282 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:48.522282 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:48.522282 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:48.522282 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:48.522282 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:48.522282 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:48.522282 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 01:00:48.522282 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 01:00:48.522282 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 01:00:48.522282 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:48.522282 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 346/714] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 01:00:48 2022] GET /identity/v3/services/network => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:48.523063 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a82847-a8d0-4221-a6c2-490925632151 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:48.589399 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a414c1db-731d-4626-ae59-0a0c4358b25c None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:48.638661 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6e3b89-6aab-4cd3-bec2-1204c0857047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:48.639522 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e3b89-6aab-4cd3-bec2-1204c0857047 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:48.639816 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e3b89-6aab-4cd3-bec2-1204c0857047 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:48.640095 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e3b89-6aab-4cd3-bec2-1204c0857047 None admin] PATH_INFO: `/v3/services` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:48.640795 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6e3b89-6aab-4cd3-bec2-1204c0857047 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:48.641521 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6e3b89-6aab-4cd3-bec2-1204c0857047 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:48.643172 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6e3b89-6aab-4cd3-bec2-1204c0857047 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:48.650199 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 369/715] 10.222.0.42 () {56 vars in 1124 bytes} [Thu Sep 8 01:00:48 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:48.650728 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a82847-a8d0-4221-a6c2-490925632151 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:48.651136 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6e3b89-6aab-4cd3-bec2-1204c0857047 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:48.681367 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:48.691464 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0244a7-a0b9-43b7-bb2c-555b66cafc27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:48.692470 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0244a7-a0b9-43b7-bb2c-555b66cafc27 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:48.692982 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0244a7-a0b9-43b7-bb2c-555b66cafc27 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:48.693447 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0244a7-a0b9-43b7-bb2c-555b66cafc27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:48.694539 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0244a7-a0b9-43b7-bb2c-555b66cafc27 None admin] RBAC: Adding query filter params (role.id=34de369ec7c54715aa83a9052a26c3f2, scope.project.id=78ad2a8fb5264fb4af935e039d28c8de, user.id=ae643cfdcb164aa9b474747cd21aa116) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:48.695088 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0244a7-a0b9-43b7-bb2c-555b66cafc27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:48.696903 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0244a7-a0b9-43b7-bb2c-555b66cafc27 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:48.704593 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 347/716] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 8 01:00:48 2022] GET /identity/v3/role_assignments?user.id=ae643cfdcb164aa9b474747cd21aa116&scope.project.id=78ad2a8fb5264fb4af935e039d28c8de&role.id=34de369ec7c54715aa83a9052a26c3f2 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:48.705683 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0244a7-a0b9-43b7-bb2c-555b66cafc27 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:48.764379 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:48.849388 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6e3b89-6aab-4cd3-bec2-1204c0857047 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:48.911773 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0244a7-a0b9-43b7-bb2c-555b66cafc27 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:48.918733 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d9dd7f-1e22-4dbc-9a0a-a8c40dc77d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:48.922882 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9dd7f-1e22-4dbc-9a0a-a8c40dc77d35 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:48.922882 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9dd7f-1e22-4dbc-9a0a-a8c40dc77d35 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:48.922882 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9dd7f-1e22-4dbc-9a0a-a8c40dc77d35 None admin] PATH_INFO: `/v3/services` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:48.922882 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d9dd7f-1e22-4dbc-9a0a-a8c40dc77d35 None admin] RBAC: Adding query filter params (name=network) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:48.922882 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d9dd7f-1e22-4dbc-9a0a-a8c40dc77d35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:48.922882 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d9dd7f-1e22-4dbc-9a0a-a8c40dc77d35 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:48.937199 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 370/717] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Sep 8 01:00:48 2022] GET /identity/v3/services?name=network => generated 124 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:48.942654 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d9dd7f-1e22-4dbc-9a0a-a8c40dc77d35 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:48.989525 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d46b3a-100e-4a7d-b4fb-a6bf061b433e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:48.990157 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d46b3a-100e-4a7d-b4fb-a6bf061b433e None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:48.990441 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d46b3a-100e-4a7d-b4fb-a6bf061b433e None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:48.990716 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d46b3a-100e-4a7d-b4fb-a6bf061b433e None admin] PATH_INFO: `/v3/services` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:48.991363 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d46b3a-100e-4a7d-b4fb-a6bf061b433e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:48.991670 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d46b3a-100e-4a7d-b4fb-a6bf061b433e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:48.993130 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d46b3a-100e-4a7d-b4fb-a6bf061b433e None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:49.000162 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 348/718] 10.222.0.42 () {56 vars in 1124 bytes} [Thu Sep 8 01:00:48 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:49.097298 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:49.132824 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d9dd7f-1e22-4dbc-9a0a-a8c40dc77d35 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:49.187170 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3664af9e-8219-450c-b45e-3f1d5c66e318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:49.187953 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3664af9e-8219-450c-b45e-3f1d5c66e318 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:49.188360 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3664af9e-8219-450c-b45e-3f1d5c66e318 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:49.188731 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3664af9e-8219-450c-b45e-3f1d5c66e318 None admin] PATH_INFO: `/v3/services` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:49.189472 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3664af9e-8219-450c-b45e-3f1d5c66e318 None admin] RBAC: Adding query filter params (type=network) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:49.189895 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3664af9e-8219-450c-b45e-3f1d5c66e318 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:49.191381 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3664af9e-8219-450c-b45e-3f1d5c66e318 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:49.195565 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83d6a03-783c-4e48-a2d9-ce36fab5e656 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:49.196078 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83d6a03-783c-4e48-a2d9-ce36fab5e656 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:49.196561 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83d6a03-783c-4e48-a2d9-ce36fab5e656 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:49.198165 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 371/719] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Sep 8 01:00:48 2022] GET /identity/v3/services?type=network => generated 124 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:00:49.198491 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 349/720] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:00:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:00:49.204822 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73fdbc7-8beb-4fda-8901-d7bc3a96fd7a None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:49.205357 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73fdbc7-8beb-4fda-8901-d7bc3a96fd7a None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:49.205806 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73fdbc7-8beb-4fda-8901-d7bc3a96fd7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:49.226476 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-f73fdbc7-8beb-4fda-8901-d7bc3a96fd7a None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:00:49.228101 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-f73fdbc7-8beb-4fda-8901-d7bc3a96fd7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:00:49.318923 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 372/721] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:00:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 01:00:49.627547 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83d6a03-783c-4e48-a2d9-ce36fab5e656 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:49.741162 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:49.753405 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83d6a03-783c-4e48-a2d9-ce36fab5e656 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:49.799864 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e159bc-4ed7-41a5-a029-c6cba3a14ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:49.800523 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e159bc-4ed7-41a5-a029-c6cba3a14ab1 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:49.800797 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e159bc-4ed7-41a5-a029-c6cba3a14ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:49.801067 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e159bc-4ed7-41a5-a029-c6cba3a14ab1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:49.816650 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e159bc-4ed7-41a5-a029-c6cba3a14ab1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:00:49.821569 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-d6e159bc-4ed7-41a5-a029-c6cba3a14ab1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 01:00:49.821569 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:49.821569 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:49.821569 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:49.821569 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:49.821569 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:49.821569 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:49.821569 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:49.821569 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:49.821569 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:49.821569 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:49.821569 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:49.821569 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:00:49.821569 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:00:49.821569 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:00:49.821569 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:00:49.822497 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:00:49.822497 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:00:49.822497 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:00:49.822497 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:00:49.822497 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:49.822497 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:49.822497 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:49.822497 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:49.822497 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:49.822497 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:49.822497 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:49.822497 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:49.822497 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:49.822497 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:49.822497 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:49.822497 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:49.823325 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:49.823325 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:49.823325 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:49.823325 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:49.823325 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:00:49.823325 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:00:49.823325 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:00:49.823325 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:00:49.823325 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:00:49.823325 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:00:49.823325 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 01:00:49.823325 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:49.823325 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 350/722] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 8 01:00:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:49.828898 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73fdbc7-8beb-4fda-8901-d7bc3a96fd7a None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:49.960781 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:49.985641 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73fdbc7-8beb-4fda-8901-d7bc3a96fd7a None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:50.034698 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3589cf-9573-4881-8cd1-89062641d065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:50.035312 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3589cf-9573-4881-8cd1-89062641d065 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:50.035589 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3589cf-9573-4881-8cd1-89062641d065 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:50.035861 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3589cf-9573-4881-8cd1-89062641d065 None admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:50.036551 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3589cf-9573-4881-8cd1-89062641d065 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:50.036852 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3589cf-9573-4881-8cd1-89062641d065 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:50.038244 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3589cf-9573-4881-8cd1-89062641d065 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:50.051943 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 373/723] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 01:00:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:50.057189 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e159bc-4ed7-41a5-a029-c6cba3a14ab1 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:50.206840 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e159bc-4ed7-41a5-a029-c6cba3a14ab1 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:50.255746 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0cec38-f472-4393-a25e-3c56bd7082f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:50.256433 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0cec38-f472-4393-a25e-3c56bd7082f3 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:50.256713 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0cec38-f472-4393-a25e-3c56bd7082f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:50.256985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0cec38-f472-4393-a25e-3c56bd7082f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:50.274427 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0cec38-f472-4393-a25e-3c56bd7082f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:50.275899 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0cec38-f472-4393-a25e-3c56bd7082f3 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:50.292250 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b75d96-8c64-48b4-a5a7-c9b94620d04b None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:50.293182 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b75d96-8c64-48b4-a5a7-c9b94620d04b None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:50.293617 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b75d96-8c64-48b4-a5a7-c9b94620d04b None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:50.295396 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 374/724] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:00:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 01:00:50.296049 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-fb0cec38-f472-4393-a25e-3c56bd7082f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:50.296049 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:50.296049 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:50.296049 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:50.296049 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:50.296049 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:50.296049 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:50.296049 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:50.296049 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:50.296049 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:50.296049 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:50.296049 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:50.296049 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:00:50.296049 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:00:50.296049 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:00:50.296049 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:00:50.297061 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:50.297061 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:50.297061 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:50.297061 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:50.297061 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:50.297061 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:50.297061 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:50.297061 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:50.297061 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:50.297061 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:50.297061 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:50.297061 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:50.297061 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:50.297061 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:50.297061 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:50.297061 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:50.297061 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:00:50.297958 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:00:50.297958 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:50.297958 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:50.297958 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 351/725] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:00:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:50.300446 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b273e37d-9785-42ef-81a7-01e4739ee517 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:50.300734 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b273e37d-9785-42ef-81a7-01e4739ee517 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:50.301015 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b273e37d-9785-42ef-81a7-01e4739ee517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:50.302641 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0cec38-f472-4393-a25e-3c56bd7082f3 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:50.309289 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:50.325634 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-b273e37d-9785-42ef-81a7-01e4739ee517 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:00:50.327111 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-b273e37d-9785-42ef-81a7-01e4739ee517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:00:50.432927 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 375/726] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:00:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 01:00:50.472576 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0cec38-f472-4393-a25e-3c56bd7082f3 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:50.507907 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b43c98e-f29e-4e13-85b2-d16b231841cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:50.508484 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43c98e-f29e-4e13-85b2-d16b231841cc None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:50.508756 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43c98e-f29e-4e13-85b2-d16b231841cc None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:50.509031 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43c98e-f29e-4e13-85b2-d16b231841cc None admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:50.509676 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b43c98e-f29e-4e13-85b2-d16b231841cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:50.509981 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b43c98e-f29e-4e13-85b2-d16b231841cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:50.511889 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b43c98e-f29e-4e13-85b2-d16b231841cc None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:50.530493 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 352/727] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:00:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:00:50.537342 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b273e37d-9785-42ef-81a7-01e4739ee517 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:50.640507 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:50.668813 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b273e37d-9785-42ef-81a7-01e4739ee517 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:50.717181 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d3302d-a5e0-4bae-99f1-0ef35d016e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:50.717877 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d3302d-a5e0-4bae-99f1-0ef35d016e06 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:50.718160 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d3302d-a5e0-4bae-99f1-0ef35d016e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:50.718430 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d3302d-a5e0-4bae-99f1-0ef35d016e06 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:50.727656 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d3302d-a5e0-4bae-99f1-0ef35d016e06 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:50.729204 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d3302d-a5e0-4bae-99f1-0ef35d016e06 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:50.740804 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-00d3302d-a5e0-4bae-99f1-0ef35d016e06 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 08 01:00:50.740804 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:50.740804 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:50.740804 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:50.740804 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:50.740804 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:50.740804 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:50.740804 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:50.740804 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:50.740804 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:50.740804 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:50.740804 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:50.740804 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 01:00:50.740804 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 01:00:50.740804 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 01:00:50.740804 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 01:00:50.741778 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:50.741778 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:50.741778 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 01:00:50.741778 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:00:50.741778 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 01:00:50.741778 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:00:50.741778 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:50.741778 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:50.741778 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:50.741778 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:50.741778 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:50.741778 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:50.741778 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:50.741778 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:50.741778 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:50.741778 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:50.741778 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:50.741778 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:50.742725 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:50.742725 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:50.742725 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 01:00:50.742725 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 01:00:50.742725 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 01:00:50.742725 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 01:00:50.742725 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 01:00:50.742725 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 01:00:50.742725 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 08 01:00:50.742725 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:50.742725 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 376/728] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 8 01:00:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:50.752207 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b43c98e-f29e-4e13-85b2-d16b231841cc None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:50.829488 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d3302d-a5e0-4bae-99f1-0ef35d016e06 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:50.838132 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:50.896209 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b43c98e-f29e-4e13-85b2-d16b231841cc None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:50.943845 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104ff690-1147-49dd-ab5e-03ca6c8d8e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:50.944473 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ff690-1147-49dd-ab5e-03ca6c8d8e2e None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:50.944748 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ff690-1147-49dd-ab5e-03ca6c8d8e2e None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:50.945017 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ff690-1147-49dd-ab5e-03ca6c8d8e2e None admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:50.945748 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104ff690-1147-49dd-ab5e-03ca6c8d8e2e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:50.946047 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104ff690-1147-49dd-ab5e-03ca6c8d8e2e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:50.947478 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104ff690-1147-49dd-ab5e-03ca6c8d8e2e None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:50.961846 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d3302d-a5e0-4bae-99f1-0ef35d016e06 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:50.982707 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 353/729] 10.222.0.42 () {56 vars in 1135 bytes} [Thu Sep 8 01:00:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:50.989783 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104ff690-1147-49dd-ab5e-03ca6c8d8e2e None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:51.001755 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:51.013323 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be2239-68c4-40d3-a304-54acdaf02f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:51.013999 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be2239-68c4-40d3-a304-54acdaf02f74 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:51.014330 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be2239-68c4-40d3-a304-54acdaf02f74 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:51.014596 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be2239-68c4-40d3-a304-54acdaf02f74 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:51.026318 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1be2239-68c4-40d3-a304-54acdaf02f74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:00:51.033008 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-c1be2239-68c4-40d3-a304-54acdaf02f74 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 01:00:51.033008 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:51.033008 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:51.033008 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:51.033008 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:51.033008 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:51.033008 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:51.033008 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:51.033008 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:51.033008 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:51.033008 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:51.033008 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:51.033008 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:00:51.033008 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:00:51.033008 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:00:51.033008 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:00:51.033995 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:00:51.033995 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:00:51.033995 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:00:51.033995 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:00:51.033995 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:51.033995 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:51.033995 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:51.033995 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:51.033995 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:51.033995 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:51.033995 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:51.033995 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:51.033995 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:51.033995 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:51.033995 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:51.033995 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:51.034924 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:51.034924 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:51.034924 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:51.034924 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:51.034924 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:00:51.034924 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:00:51.034924 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:00:51.034924 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:00:51.034924 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:00:51.034924 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:00:51.034924 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 01:00:51.034924 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:51.034924 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 377/730] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 8 01:00:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 01:00:51.041737 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be2239-68c4-40d3-a304-54acdaf02f74 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:51.171701 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104ff690-1147-49dd-ab5e-03ca6c8d8e2e None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:51.188080 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be2239-68c4-40d3-a304-54acdaf02f74 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:51.222741 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f6c5de-7a30-4cc0-8ceb-56051f2ee02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:51.223352 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f6c5de-7a30-4cc0-8ceb-56051f2ee02d None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:51.223634 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f6c5de-7a30-4cc0-8ceb-56051f2ee02d None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:51.223907 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f6c5de-7a30-4cc0-8ceb-56051f2ee02d None admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:51.224608 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f6c5de-7a30-4cc0-8ceb-56051f2ee02d None admin] RBAC: Adding query filter params (name=service) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:51.224913 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f6c5de-7a30-4cc0-8ceb-56051f2ee02d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:51.225789 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:51.226362 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f6c5de-7a30-4cc0-8ceb-56051f2ee02d None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:51.235422 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 378/731] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 8 01:00:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:51.241001 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f6c5de-7a30-4cc0-8ceb-56051f2ee02d None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:51.257146 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5972d3-92f2-4730-9412-de8b696287d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:51.257775 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5972d3-92f2-4730-9412-de8b696287d8 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:51.258055 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5972d3-92f2-4730-9412-de8b696287d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:51.258330 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5972d3-92f2-4730-9412-de8b696287d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:51.337164 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:51.374475 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5972d3-92f2-4730-9412-de8b696287d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:51.375942 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5972d3-92f2-4730-9412-de8b696287d8 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:51.443191 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f6c5de-7a30-4cc0-8ceb-56051f2ee02d None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:51.488829 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 354/732] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 01:00:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 01:00:51.492503 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b083f0d-85d1-4952-98a8-aa37987a873f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:51.493365 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b083f0d-85d1-4952-98a8-aa37987a873f None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:51.493961 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b083f0d-85d1-4952-98a8-aa37987a873f None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:51.494411 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b083f0d-85d1-4952-98a8-aa37987a873f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:51.494743 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da14dd-6bf1-4a69-99fe-43befded9d51 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:51.495120 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da14dd-6bf1-4a69-99fe-43befded9d51 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:51.495482 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da14dd-6bf1-4a69-99fe-43befded9d51 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:51.501870 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 355/733] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:00:51 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:00:51.501870 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09da14dd-6bf1-4a69-99fe-43befded9d51 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:51.515246 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b083f0d-85d1-4952-98a8-aa37987a873f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:51.516943 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b083f0d-85d1-4952-98a8-aa37987a873f None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:51.529496 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:51.532738 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-5b083f0d-85d1-4952-98a8-aa37987a873f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:51.532738 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:51.532738 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:51.532738 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:51.532738 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:51.532738 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:51.532738 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:51.532738 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:51.532738 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:51.532738 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:51.532738 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:51.532738 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:51.532738 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:00:51.532738 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:00:51.532738 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:00:51.532738 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:00:51.533835 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:51.533835 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:51.533835 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:51.533835 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:51.533835 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:51.533835 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:51.533835 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:51.533835 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:51.533835 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:51.533835 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:51.533835 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:51.533835 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:51.533835 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:51.533835 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:51.533835 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:51.533835 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:51.533835 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:00:51.534777 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:00:51.534777 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:51.534777 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:51.540177 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 379/734] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:00:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:51.540177 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8838b019-0f66-4663-b712-b4d049955fff None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:51.540177 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8838b019-0f66-4663-b712-b4d049955fff None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:51.540177 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8838b019-0f66-4663-b712-b4d049955fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:51.557417 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-8838b019-0f66-4663-b712-b4d049955fff None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:00:51.558797 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-8838b019-0f66-4663-b712-b4d049955fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:00:51.644176 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 380/735] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:00:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 01:00:51.644176 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8838b019-0f66-4663-b712-b4d049955fff None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:51.662941 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09da14dd-6bf1-4a69-99fe-43befded9d51 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:51.701239 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:51.734512 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d40a014-0651-4e8a-8aec-119ac088371e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:51.735194 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d40a014-0651-4e8a-8aec-119ac088371e None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:51.735473 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d40a014-0651-4e8a-8aec-119ac088371e None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:51.735742 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d40a014-0651-4e8a-8aec-119ac088371e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:51.751111 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d40a014-0651-4e8a-8aec-119ac088371e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:51.752701 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d40a014-0651-4e8a-8aec-119ac088371e None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:51.768736 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-3d40a014-0651-4e8a-8aec-119ac088371e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:51.768736 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:51.768736 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:51.768736 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:51.768736 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:51.768736 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:51.768736 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:51.768736 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:51.768736 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:51.768736 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:51.768736 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:51.768736 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:51.768736 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:00:51.768736 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:00:51.768736 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:00:51.768736 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:00:51.769709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:51.769709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:51.769709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:51.769709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:51.769709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:51.769709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:51.769709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:51.769709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:51.769709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:51.769709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:51.769709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:51.769709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:51.769709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:51.769709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:51.769709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:51.769709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:51.769709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:00:51.770605 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:00:51.770605 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:51.770605 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:51.770605 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 356/736] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:00:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:51.775640 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d40a014-0651-4e8a-8aec-119ac088371e None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:51.856197 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8838b019-0f66-4663-b712-b4d049955fff None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:51.894979 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:51.895936 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0662b87f-6dd4-400e-8188-5fd87f75cd77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:51.896631 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0662b87f-6dd4-400e-8188-5fd87f75cd77 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:51.896905 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0662b87f-6dd4-400e-8188-5fd87f75cd77 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:51.897222 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0662b87f-6dd4-400e-8188-5fd87f75cd77 None admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:51.898080 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0662b87f-6dd4-400e-8188-5fd87f75cd77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:51.898441 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0662b87f-6dd4-400e-8188-5fd87f75cd77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:51.900084 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0662b87f-6dd4-400e-8188-5fd87f75cd77 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:51.912552 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 381/737] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:00:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:51.913781 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cb4b9c-a056-4331-b835-821a5aeac6a4 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:51.914065 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cb4b9c-a056-4331-b835-821a5aeac6a4 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:51.914393 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cb4b9c-a056-4331-b835-821a5aeac6a4 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:51.915830 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 382/738] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:00:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:00:51.919531 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cb4b9c-a056-4331-b835-821a5aeac6a4 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:51.925201 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d40a014-0651-4e8a-8aec-119ac088371e None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:51.969795 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbc02f8-fe9b-4656-abe6-1cc41afe7168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:51.970636 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc02f8-fe9b-4656-abe6-1cc41afe7168 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:51.971078 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc02f8-fe9b-4656-abe6-1cc41afe7168 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:51.971361 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc02f8-fe9b-4656-abe6-1cc41afe7168 None admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:51.972009 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbc02f8-fe9b-4656-abe6-1cc41afe7168 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:51.972347 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbc02f8-fe9b-4656-abe6-1cc41afe7168 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:51.973727 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbc02f8-fe9b-4656-abe6-1cc41afe7168 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:51.986495 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 357/739] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:00:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:51.987680 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750001cc-99f5-4bd2-bdf1-fb687509e806 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:51.987967 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750001cc-99f5-4bd2-bdf1-fb687509e806 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:51.988276 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750001cc-99f5-4bd2-bdf1-fb687509e806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:52.006201 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-750001cc-99f5-4bd2-bdf1-fb687509e806 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:00:52.007446 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-750001cc-99f5-4bd2-bdf1-fb687509e806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:00:52.021021 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:52.031978 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cb4b9c-a056-4331-b835-821a5aeac6a4 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:52.066282 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13ded9d-c7d0-4d1c-a0f6-45c4ba0b9307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:52.066904 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ded9d-c7d0-4d1c-a0f6-45c4ba0b9307 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:52.067188 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ded9d-c7d0-4d1c-a0f6-45c4ba0b9307 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:52.067452 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ded9d-c7d0-4d1c-a0f6-45c4ba0b9307 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:52.074845 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 358/740] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:00:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 01:00:52.075114 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13ded9d-c7d0-4d1c-a0f6-45c4ba0b9307 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:52.076090 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee976e-bb8b-4079-b051-881aa6152e8e None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:52.076405 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee976e-bb8b-4079-b051-881aa6152e8e None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:52.076669 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13ded9d-c7d0-4d1c-a0f6-45c4ba0b9307 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:52.076768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee976e-bb8b-4079-b051-881aa6152e8e None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:52.080285 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 359/741] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:00:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:00:52.080285 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee976e-bb8b-4079-b051-881aa6152e8e None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:52.101420 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-e13ded9d-c7d0-4d1c-a0f6-45c4ba0b9307 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 01:00:52.101420 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:52.101420 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:52.101420 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:52.101420 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:52.101420 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:52.101420 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:52.101420 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:52.101420 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:52.101420 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:52.101420 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:52.101420 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:52.101420 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 01:00:52.101420 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 01:00:52.101420 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 01:00:52.101420 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 01:00:52.102586 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:52.102586 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:52.102586 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 01:00:52.102586 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:00:52.102586 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 01:00:52.102586 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:00:52.102586 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:52.102586 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:52.102586 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:52.102586 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:52.102586 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:52.102586 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:52.102586 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:52.102586 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:52.102586 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:52.102586 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:52.102586 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:52.102586 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:52.103662 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:52.103662 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:52.103662 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 01:00:52.103662 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 01:00:52.103662 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 01:00:52.103662 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 01:00:52.103662 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 01:00:52.103662 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 01:00:52.103662 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 01:00:52.103662 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:52.103662 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 383/742] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Sep 8 01:00:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:52.103662 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13ded9d-c7d0-4d1c-a0f6-45c4ba0b9307 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:52.153394 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:52.200627 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee976e-bb8b-4079-b051-881aa6152e8e None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:52.211533 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13ded9d-c7d0-4d1c-a0f6-45c4ba0b9307 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:52.238506 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aa2644-98dd-4b05-9631-714e1e6838d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:52.239328 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aa2644-98dd-4b05-9631-714e1e6838d2 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:52.239773 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aa2644-98dd-4b05-9631-714e1e6838d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:52.240281 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aa2644-98dd-4b05-9631-714e1e6838d2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:52.245913 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bb2c20-90ad-444c-a80f-00f3400cbb01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:52.246544 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb2c20-90ad-444c-a80f-00f3400cbb01 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:52.246828 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb2c20-90ad-444c-a80f-00f3400cbb01 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:52.247102 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb2c20-90ad-444c-a80f-00f3400cbb01 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:52.252249 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aa2644-98dd-4b05-9631-714e1e6838d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:00:52.253702 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bb2c20-90ad-444c-a80f-00f3400cbb01 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:52.255183 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bb2c20-90ad-444c-a80f-00f3400cbb01 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:52.258278 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-40aa2644-98dd-4b05-9631-714e1e6838d2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 01:00:52.258278 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:52.258278 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:52.258278 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:52.258278 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:52.258278 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:52.258278 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:52.258278 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:52.258278 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:52.258278 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:52.258278 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:52.258278 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:52.258278 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:00:52.258278 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:00:52.258278 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:00:52.258278 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:00:52.259412 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:00:52.259412 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:00:52.259412 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:00:52.259412 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:00:52.259412 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:52.259412 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:52.259412 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:52.259412 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:52.259412 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:52.259412 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:52.259412 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:52.259412 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:52.259412 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:52.259412 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:52.259412 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:52.259412 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:52.260504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:52.260504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:52.260504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:52.260504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:52.260504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:00:52.260504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:00:52.260504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:00:52.260504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:00:52.260504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:00:52.260504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:00:52.260504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 01:00:52.260504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:52.263681 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 360/743] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 8 01:00:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:52.265045 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b5dda-a493-40d5-a0bf-f12d0993df22 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:52.265458 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b5dda-a493-40d5-a0bf-f12d0993df22 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:52.265858 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b5dda-a493-40d5-a0bf-f12d0993df22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:52.267115 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-00bb2c20-90ad-444c-a80f-00f3400cbb01 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:00:52.267115 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:52.267115 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:52.267115 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:52.267115 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:52.267115 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:52.267115 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:52.267115 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:52.267115 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:52.267115 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:52.267115 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:52.267115 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:52.267115 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 01:00:52.267115 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 01:00:52.267115 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 01:00:52.268044 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 01:00:52.268044 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:52.268044 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:52.268044 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:52.268044 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:52.268044 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:52.268044 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:52.268044 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:52.268044 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:52.268044 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:52.268044 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:52.268044 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:52.268044 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:52.268044 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:52.268044 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:52.268044 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:52.268044 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:52.269024 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 01:00:52.269024 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 01:00:52.269024 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 01:00:52.269024 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 01:00:52.269024 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 01:00:52.269024 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 01:00:52.269024 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:00:52.269024 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:52.272838 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 384/744] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 01:00:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:52.273542 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bb2c20-90ad-444c-a80f-00f3400cbb01 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:52.287348 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:52.288190 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-601b5dda-a493-40d5-a0bf-f12d0993df22 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:00:52.289522 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-601b5dda-a493-40d5-a0bf-f12d0993df22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:00:52.361296 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 361/745] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:00:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 01:00:52.362205 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601b5dda-a493-40d5-a0bf-f12d0993df22 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:52.391738 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bb2c20-90ad-444c-a80f-00f3400cbb01 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:52.421535 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:52.429496 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5363b7c-f70a-4a68-a5a3-a04bf8126031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:52.430106 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5363b7c-f70a-4a68-a5a3-a04bf8126031 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:52.430377 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5363b7c-f70a-4a68-a5a3-a04bf8126031 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:52.430640 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5363b7c-f70a-4a68-a5a3-a04bf8126031 None admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:52.431358 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5363b7c-f70a-4a68-a5a3-a04bf8126031 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:52.431655 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5363b7c-f70a-4a68-a5a3-a04bf8126031 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:52.434621 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5363b7c-f70a-4a68-a5a3-a04bf8126031 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:52.459121 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 385/746] 10.222.0.42 () {56 vars in 1149 bytes} [Thu Sep 8 01:00:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:52.459984 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5363b7c-f70a-4a68-a5a3-a04bf8126031 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:52.469800 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601b5dda-a493-40d5-a0bf-f12d0993df22 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:52.503328 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3f4ee4-b19c-496f-901d-97deab499fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:52.503931 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f4ee4-b19c-496f-901d-97deab499fab None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:52.504247 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f4ee4-b19c-496f-901d-97deab499fab None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:52.504521 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f4ee4-b19c-496f-901d-97deab499fab None admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:52.505188 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3f4ee4-b19c-496f-901d-97deab499fab None admin] RBAC: Adding query filter params (name=admin) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:52.505491 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3f4ee4-b19c-496f-901d-97deab499fab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:52.506862 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3f4ee4-b19c-496f-901d-97deab499fab None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:52.520852 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 362/747] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 01:00:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:52.521555 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3f4ee4-b19c-496f-901d-97deab499fab None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:52.532687 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:52.569990 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5363b7c-f70a-4a68-a5a3-a04bf8126031 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:52.605608 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93057a6-3001-4b6c-ae9c-5c8ffeb1db14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:52.606219 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93057a6-3001-4b6c-ae9c-5c8ffeb1db14 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:52.606491 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93057a6-3001-4b6c-ae9c-5c8ffeb1db14 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:52.606761 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93057a6-3001-4b6c-ae9c-5c8ffeb1db14 None admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:52.607450 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93057a6-3001-4b6c-ae9c-5c8ffeb1db14 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:52.607804 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93057a6-3001-4b6c-ae9c-5c8ffeb1db14 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:52.609216 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93057a6-3001-4b6c-ae9c-5c8ffeb1db14 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:52.620999 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 386/748] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 8 01:00:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:52.621895 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93057a6-3001-4b6c-ae9c-5c8ffeb1db14 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:52.635744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3f4ee4-b19c-496f-901d-97deab499fab None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:52.653537 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:52.671587 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b3dfd8-816f-4e55-ad96-53ad95abc10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:52.672229 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3dfd8-816f-4e55-ad96-53ad95abc10e None admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:52.672512 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3dfd8-816f-4e55-ad96-53ad95abc10e None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:52.672790 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3dfd8-816f-4e55-ad96-53ad95abc10e None admin] PATH_INFO: `/v3/services` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:52.673433 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b3dfd8-816f-4e55-ad96-53ad95abc10e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:52.674809 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b3dfd8-816f-4e55-ad96-53ad95abc10e None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:52.690924 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 363/749] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Sep 8 01:00:52 2022] POST /identity/v3/services => generated 251 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:00:52.691725 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b3dfd8-816f-4e55-ad96-53ad95abc10e None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:52.739795 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93057a6-3001-4b6c-ae9c-5c8ffeb1db14 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:52.770573 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:52.794743 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528dd2c-04b3-4746-a0fb-3735193d4713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:52.795374 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528dd2c-04b3-4746-a0fb-3735193d4713 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:52.795653 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528dd2c-04b3-4746-a0fb-3735193d4713 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:52.795921 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528dd2c-04b3-4746-a0fb-3735193d4713 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:52.814091 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b3dfd8-816f-4e55-ad96-53ad95abc10e None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:52.856862 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a4652-c997-46b9-96eb-1399b12facb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:52.857501 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a4652-c997-46b9-96eb-1399b12facb9 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:52.857783 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a4652-c997-46b9-96eb-1399b12facb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:52.858058 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a4652-c997-46b9-96eb-1399b12facb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:52.873085 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e528dd2c-04b3-4746-a0fb-3735193d4713 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:52.873605 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45a4652-c997-46b9-96eb-1399b12facb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:52.874563 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e528dd2c-04b3-4746-a0fb-3735193d4713 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:52.874706 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45a4652-c997-46b9-96eb-1399b12facb9 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:52.893026 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-b45a4652-c997-46b9-96eb-1399b12facb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:52.893026 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:52.893026 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:52.893026 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:52.893026 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:52.893026 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:52.893026 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:52.893026 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:52.893026 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:52.893026 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:52.893026 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:52.893026 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:52.893026 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:00:52.893026 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:00:52.893026 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:00:52.893026 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:00:52.894176 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:52.894176 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:52.894176 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:52.894176 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:52.894176 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:52.894176 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:52.894176 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:52.894176 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:52.894176 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:52.894176 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:52.894176 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:52.894176 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:52.894176 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:52.894176 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:52.894176 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:52.894176 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:52.894176 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:00:52.895080 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:00:52.895080 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:52.895080 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:52.895080 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 364/750] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:00:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:52.896071 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a4652-c997-46b9-96eb-1399b12facb9 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:52.930223 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:52.989382 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 387/751] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 01:00:52 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 01:00:52.990452 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528dd2c-04b3-4746-a0fb-3735193d4713 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:53.020316 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:53.020704 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a4652-c997-46b9-96eb-1399b12facb9 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:53.056338 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d562042-1a3d-4463-ae63-c9e511521e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:53.057070 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d562042-1a3d-4463-ae63-c9e511521e22 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:53.057421 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d562042-1a3d-4463-ae63-c9e511521e22 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:53.057770 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d562042-1a3d-4463-ae63-c9e511521e22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:53.058665 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d562042-1a3d-4463-ae63-c9e511521e22 None admin] RBAC: Adding query filter params (role.id=34de369ec7c54715aa83a9052a26c3f2, scope.project.id=78ad2a8fb5264fb4af935e039d28c8de, user.id=8add26872fe6479aa9d7de155a17195a) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:53.059084 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d562042-1a3d-4463-ae63-c9e511521e22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:53.060564 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d562042-1a3d-4463-ae63-c9e511521e22 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:53.066159 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 365/752] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 8 01:00:52 2022] GET /identity/v3/role_assignments?user.id=8add26872fe6479aa9d7de155a17195a&scope.project.id=78ad2a8fb5264fb4af935e039d28c8de&role.id=34de369ec7c54715aa83a9052a26c3f2 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:53.067442 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d562042-1a3d-4463-ae63-c9e511521e22 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:53.107076 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528dd2c-04b3-4746-a0fb-3735193d4713 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:53.144066 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566ee8c8-edf9-4f93-b504-56336a6bc417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:53.144909 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ee8c8-edf9-4f93-b504-56336a6bc417 None admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:53.145256 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ee8c8-edf9-4f93-b504-56336a6bc417 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:53.145592 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ee8c8-edf9-4f93-b504-56336a6bc417 None admin] PATH_INFO: `/v3/services` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:53.146266 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566ee8c8-edf9-4f93-b504-56336a6bc417 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:53.147731 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566ee8c8-edf9-4f93-b504-56336a6bc417 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:53.166566 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 388/753] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Sep 8 01:00:52 2022] POST /identity/v3/services => generated 240 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:00:53.168217 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566ee8c8-edf9-4f93-b504-56336a6bc417 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:53.198462 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:53.213264 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d562042-1a3d-4463-ae63-c9e511521e22 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:53.286175 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:53.302688 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2001a7b5-6fd4-4edb-874e-dcb9dc4f9042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:53.303416 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2001a7b5-6fd4-4edb-874e-dcb9dc4f9042 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:53.303698 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2001a7b5-6fd4-4edb-874e-dcb9dc4f9042 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:53.304024 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2001a7b5-6fd4-4edb-874e-dcb9dc4f9042 None admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:53.304776 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2001a7b5-6fd4-4edb-874e-dcb9dc4f9042 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:53.305078 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2001a7b5-6fd4-4edb-874e-dcb9dc4f9042 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:53.307946 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566ee8c8-edf9-4f93-b504-56336a6bc417 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:53.309284 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2001a7b5-6fd4-4edb-874e-dcb9dc4f9042 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:53.328788 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 366/754] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:00:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:53.335221 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2001a7b5-6fd4-4edb-874e-dcb9dc4f9042 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:53.351659 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117eb856-da35-4fa3-bc7e-45275ea3748e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:53.352346 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117eb856-da35-4fa3-bc7e-45275ea3748e None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:53.352629 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117eb856-da35-4fa3-bc7e-45275ea3748e None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:53.352897 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117eb856-da35-4fa3-bc7e-45275ea3748e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:53.384043 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117eb856-da35-4fa3-bc7e-45275ea3748e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:53.385617 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117eb856-da35-4fa3-bc7e-45275ea3748e None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:53.420324 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-117eb856-da35-4fa3-bc7e-45275ea3748e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:53.420324 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:53.420324 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:53.420324 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:53.420324 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:53.420324 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:53.420324 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:53.420324 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:53.420324 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:53.420324 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:53.420324 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:53.420324 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:53.420324 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:00:53.420324 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:00:53.420324 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:00:53.420324 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:00:53.421339 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:53.421339 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:53.421339 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:53.421339 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:53.421339 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:53.421339 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:53.421339 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:53.421339 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:53.421339 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:53.421339 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:53.421339 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:53.421339 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:53.421339 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:53.421339 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:53.421339 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:53.421339 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:53.421339 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:00:53.422235 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:00:53.422235 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:53.422235 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:53.422235 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 389/755] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:00:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:53.427494 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117eb856-da35-4fa3-bc7e-45275ea3748e None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:53.509361 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2001a7b5-6fd4-4edb-874e-dcb9dc4f9042 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:53.555595 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117eb856-da35-4fa3-bc7e-45275ea3748e None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:53.556533 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75408e0f-88a1-4df0-8ed5-bdc0499e5394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:53.557624 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75408e0f-88a1-4df0-8ed5-bdc0499e5394 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:53.558607 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75408e0f-88a1-4df0-8ed5-bdc0499e5394 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:53.560286 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75408e0f-88a1-4df0-8ed5-bdc0499e5394 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:53.569841 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75408e0f-88a1-4df0-8ed5-bdc0499e5394 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:53.572274 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75408e0f-88a1-4df0-8ed5-bdc0499e5394 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:53.589053 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-75408e0f-88a1-4df0-8ed5-bdc0499e5394 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 01:00:53.589053 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:53.589053 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:53.589053 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:53.589053 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:53.589053 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:53.589053 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:53.589053 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:53.589053 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:53.589053 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:53.589053 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:53.589053 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:53.589053 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 01:00:53.589053 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 01:00:53.589053 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 01:00:53.589053 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 01:00:53.590187 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:53.590187 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:53.590187 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 01:00:53.590187 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:00:53.590187 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 01:00:53.590187 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:00:53.590187 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:53.590187 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:53.590187 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:53.590187 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:53.590187 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:53.590187 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:53.590187 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:53.590187 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:53.590187 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:53.590187 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:53.590187 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:53.590187 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:53.591237 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:53.591237 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:53.591237 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 01:00:53.591237 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 01:00:53.591237 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 01:00:53.591237 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 01:00:53.591237 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 01:00:53.591237 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 01:00:53.591237 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 01:00:53.591237 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:53.591237 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 367/756] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Sep 8 01:00:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:53.595926 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75408e0f-88a1-4df0-8ed5-bdc0499e5394 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:53.598147 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854b0005-c684-40d7-b1d8-f21b63711f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:53.599109 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854b0005-c684-40d7-b1d8-f21b63711f49 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:53.599594 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854b0005-c684-40d7-b1d8-f21b63711f49 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:53.600078 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854b0005-c684-40d7-b1d8-f21b63711f49 None admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:53.601006 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854b0005-c684-40d7-b1d8-f21b63711f49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:53.601538 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854b0005-c684-40d7-b1d8-f21b63711f49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:53.603130 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854b0005-c684-40d7-b1d8-f21b63711f49 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:53.606756 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:53.608673 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:53.624280 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 390/757] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:00:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:00:53.625702 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854b0005-c684-40d7-b1d8-f21b63711f49 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:53.759985 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854b0005-c684-40d7-b1d8-f21b63711f49 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:53.810507 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75408e0f-88a1-4df0-8ed5-bdc0499e5394 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:53.828949 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd3f6dc-735c-4a4b-88ce-0f0f06d3d402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:53.829586 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3f6dc-735c-4a4b-88ce-0f0f06d3d402 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:53.829855 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3f6dc-735c-4a4b-88ce-0f0f06d3d402 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:53.830131 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3f6dc-735c-4a4b-88ce-0f0f06d3d402 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:53.844709 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd3f6dc-735c-4a4b-88ce-0f0f06d3d402 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:53.846169 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd3f6dc-735c-4a4b-88ce-0f0f06d3d402 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:53.869167 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-6cd3f6dc-735c-4a4b-88ce-0f0f06d3d402 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:00:53.869167 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:53.869167 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:53.869167 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:53.869167 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:53.869167 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:53.869167 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:53.869167 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:53.869167 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:53.869167 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:53.869167 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:53.869167 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:53.869167 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 01:00:53.869167 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 01:00:53.869167 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 01:00:53.870126 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 01:00:53.870126 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:53.870126 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:53.870126 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:53.870126 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:53.870126 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:53.870126 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:53.870126 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:53.870126 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:53.870126 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:53.870126 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:53.870126 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:53.870126 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:53.870126 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:53.870126 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:53.870126 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:53.870126 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:53.871130 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 01:00:53.871130 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 01:00:53.871130 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 01:00:53.871130 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 01:00:53.871130 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 01:00:53.871130 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 01:00:53.871130 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:00:53.871130 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:53.874356 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 391/758] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 01:00:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 01:00:53.878709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88732942-0bad-47cf-a196-687e763e03a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:53.879331 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88732942-0bad-47cf-a196-687e763e03a8 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:53.879615 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88732942-0bad-47cf-a196-687e763e03a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:53.879888 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88732942-0bad-47cf-a196-687e763e03a8 None admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:53.880663 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88732942-0bad-47cf-a196-687e763e03a8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:53.880972 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88732942-0bad-47cf-a196-687e763e03a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:53.881133 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd3f6dc-735c-4a4b-88ce-0f0f06d3d402 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:53.883188 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88732942-0bad-47cf-a196-687e763e03a8 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:53.918885 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 368/759] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Sep 8 01:00:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:00:53.925568 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88732942-0bad-47cf-a196-687e763e03a8 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:54.013507 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:54.016509 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:54.057625 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88732942-0bad-47cf-a196-687e763e03a8 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:54.096606 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aa9f08-85a7-4ad8-8bb3-23adedcae866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:54.097860 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa9f08-85a7-4ad8-8bb3-23adedcae866 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:54.098368 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa9f08-85a7-4ad8-8bb3-23adedcae866 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:54.098849 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa9f08-85a7-4ad8-8bb3-23adedcae866 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:54.111445 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aa9f08-85a7-4ad8-8bb3-23adedcae866 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:54.113745 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aa9f08-85a7-4ad8-8bb3-23adedcae866 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:54.131124 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-19aa9f08-85a7-4ad8-8bb3-23adedcae866 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:54.131124 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:54.131124 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:54.131124 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:54.131124 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:54.131124 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:54.131124 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:54.131124 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:54.131124 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:54.131124 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:54.131124 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:54.131124 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:54.131124 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:00:54.131124 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:00:54.131124 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:00:54.131124 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:00:54.132116 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:54.132116 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:54.132116 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:54.132116 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:54.132116 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:54.132116 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:54.132116 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:54.132116 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:54.132116 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:54.132116 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:54.132116 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:54.132116 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:54.132116 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:54.132116 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:54.132116 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:54.132116 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:54.132116 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:00:54.133054 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:00:54.133054 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:54.133054 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:54.133225 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd3f6dc-735c-4a4b-88ce-0f0f06d3d402 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:54.135408 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 369/760] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:00:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:54.140958 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aa9f08-85a7-4ad8-8bb3-23adedcae866 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:54.211343 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5773af0f-fe2b-4164-836a-a17f3fc9f0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:54.213166 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5773af0f-fe2b-4164-836a-a17f3fc9f0fb None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:54.215240 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5773af0f-fe2b-4164-836a-a17f3fc9f0fb None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:54.215720 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5773af0f-fe2b-4164-836a-a17f3fc9f0fb None admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:54.216665 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5773af0f-fe2b-4164-836a-a17f3fc9f0fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:54.219184 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5773af0f-fe2b-4164-836a-a17f3fc9f0fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:54.220868 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5773af0f-fe2b-4164-836a-a17f3fc9f0fb None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:54.244586 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 392/761] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 8 01:00:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:54.247562 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5773af0f-fe2b-4164-836a-a17f3fc9f0fb None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:54.327149 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aa9f08-85a7-4ad8-8bb3-23adedcae866 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:54.347776 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:54.368793 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aff0dd-e1e5-40c3-be65-2bcc7f39ba0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:54.372156 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aff0dd-e1e5-40c3-be65-2bcc7f39ba0b None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:54.372156 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aff0dd-e1e5-40c3-be65-2bcc7f39ba0b None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:54.372156 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aff0dd-e1e5-40c3-be65-2bcc7f39ba0b None admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:54.372156 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aff0dd-e1e5-40c3-be65-2bcc7f39ba0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:54.372156 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aff0dd-e1e5-40c3-be65-2bcc7f39ba0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:54.372525 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aff0dd-e1e5-40c3-be65-2bcc7f39ba0b None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:54.390080 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 370/762] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:00:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:00:54.395216 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aff0dd-e1e5-40c3-be65-2bcc7f39ba0b None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:54.439395 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5773af0f-fe2b-4164-836a-a17f3fc9f0fb None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:54.488513 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:54.514188 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998dc25d-983c-4c18-92d0-468ab4dbd8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:54.515146 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998dc25d-983c-4c18-92d0-468ab4dbd8be None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:54.515623 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998dc25d-983c-4c18-92d0-468ab4dbd8be None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:54.516090 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998dc25d-983c-4c18-92d0-468ab4dbd8be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:54.517174 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998dc25d-983c-4c18-92d0-468ab4dbd8be None admin] RBAC: Adding query filter params (role.id=609d73f56e6a48698397131168abe6df, scope.project.id=78ad2a8fb5264fb4af935e039d28c8de, user.id=de75fd7d7721433684cafa2e6d3faaae) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:54.517727 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998dc25d-983c-4c18-92d0-468ab4dbd8be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:54.519614 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998dc25d-983c-4c18-92d0-468ab4dbd8be None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:54.531110 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 393/763] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 8 01:00:54 2022] GET /identity/v3/role_assignments?user.id=de75fd7d7721433684cafa2e6d3faaae&scope.project.id=78ad2a8fb5264fb4af935e039d28c8de&role.id=609d73f56e6a48698397131168abe6df => generated 259 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:54.598474 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aff0dd-e1e5-40c3-be65-2bcc7f39ba0b None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:54.666346 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907650fe-b8ff-43c3-b48c-55fdb89f2896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:54.667004 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907650fe-b8ff-43c3-b48c-55fdb89f2896 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:54.667286 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907650fe-b8ff-43c3-b48c-55fdb89f2896 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:54.667612 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907650fe-b8ff-43c3-b48c-55fdb89f2896 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:54.693489 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907650fe-b8ff-43c3-b48c-55fdb89f2896 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:54.695006 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907650fe-b8ff-43c3-b48c-55fdb89f2896 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:54.708205 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-907650fe-b8ff-43c3-b48c-55fdb89f2896 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:00:54.708205 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:54.708205 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:54.708205 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:54.708205 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:54.708205 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:54.708205 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:54.708205 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:54.708205 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:54.708205 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:54.708205 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:54.708205 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:54.708205 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 01:00:54.708205 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 01:00:54.708205 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 01:00:54.709164 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 01:00:54.709164 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:54.709164 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:54.709164 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:54.709164 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:54.709164 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:54.709164 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:54.709164 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:54.709164 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:54.709164 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:54.709164 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:54.709164 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:54.709164 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:54.709164 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:54.709164 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:54.709164 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:54.709164 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:54.710633 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 01:00:54.710633 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 01:00:54.710633 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 01:00:54.710633 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 01:00:54.710633 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 01:00:54.710633 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 01:00:54.710633 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:00:54.710633 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:54.710633 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 371/764] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 01:00:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:54.716045 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998dc25d-983c-4c18-92d0-468ab4dbd8be None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:54.834010 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998dc25d-983c-4c18-92d0-468ab4dbd8be None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:54.869556 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfad9318-f749-470c-a051-d15b82539ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:54.871118 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad9318-f749-470c-a051-d15b82539ab2 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:54.871617 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad9318-f749-470c-a051-d15b82539ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:54.872084 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad9318-f749-470c-a051-d15b82539ab2 None admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:54.872998 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfad9318-f749-470c-a051-d15b82539ab2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:54.873528 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfad9318-f749-470c-a051-d15b82539ab2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:54.875173 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfad9318-f749-470c-a051-d15b82539ab2 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:54.888543 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 394/765] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 8 01:00:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:54.897444 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907650fe-b8ff-43c3-b48c-55fdb89f2896 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:54.962642 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:55.035907 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4c5e0e-e3d4-4baf-8f97-74459f63982f None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:55.036234 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4c5e0e-e3d4-4baf-8f97-74459f63982f None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:55.036522 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4c5e0e-e3d4-4baf-8f97-74459f63982f None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:55.038092 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 395/766] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:00:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:00:55.043423 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f65658-a58b-4a13-b7e6-8beed514be70 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:55.043712 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f65658-a58b-4a13-b7e6-8beed514be70 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:55.043989 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f65658-a58b-4a13-b7e6-8beed514be70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:55.049170 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907650fe-b8ff-43c3-b48c-55fdb89f2896 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:55.076271 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-d4f65658-a58b-4a13-b7e6-8beed514be70 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:00:55.077491 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-d4f65658-a58b-4a13-b7e6-8beed514be70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:00:55.124384 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d6c586-336a-41c7-bf49-0e0e9f16ffea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:55.124975 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d6c586-336a-41c7-bf49-0e0e9f16ffea None admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:55.125255 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d6c586-336a-41c7-bf49-0e0e9f16ffea None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:55.125529 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d6c586-336a-41c7-bf49-0e0e9f16ffea None admin] PATH_INFO: `/v3/projects/78ad2a8fb5264fb4af935e039d28c8de/users/ae643cfdcb164aa9b474747cd21aa116/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:55.128843 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d6c586-336a-41c7-bf49-0e0e9f16ffea None admin] RBAC: Authorizing `identity:create_grant(project_id=78ad2a8fb5264fb4af935e039d28c8de, user_id=ae643cfdcb164aa9b474747cd21aa116, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:55.130292 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d6c586-336a-41c7-bf49-0e0e9f16ffea None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:55.144504 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:55.161286 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 372/767] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 8 01:00:54 2022] PUT /identity/v3/projects/78ad2a8fb5264fb4af935e039d28c8de/users/ae643cfdcb164aa9b474747cd21aa116/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:00:55.214973 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 396/768] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:00:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 01:00:55.649933 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d6c586-336a-41c7-bf49-0e0e9f16ffea None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:55.778837 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:55.782485 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d6c586-336a-41c7-bf49-0e0e9f16ffea None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:55.800393 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33042f3-71ab-4d67-bf98-5d732cfdcae4 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:55.800697 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33042f3-71ab-4d67-bf98-5d732cfdcae4 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:55.800937 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33042f3-71ab-4d67-bf98-5d732cfdcae4 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:55.802517 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 397/769] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:00:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:00:55.807588 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea059442-464b-44fa-921e-acb31a304d77 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:55.807871 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea059442-464b-44fa-921e-acb31a304d77 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:55.808901 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea059442-464b-44fa-921e-acb31a304d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:55.833170 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-ea059442-464b-44fa-921e-acb31a304d77 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:00:55.834414 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-ea059442-464b-44fa-921e-acb31a304d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:00:55.854029 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbd3924-8c7a-4781-866c-edd55f3ac857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:55.854693 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbd3924-8c7a-4781-866c-edd55f3ac857 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:55.855049 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbd3924-8c7a-4781-866c-edd55f3ac857 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:55.855324 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbd3924-8c7a-4781-866c-edd55f3ac857 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:55.855974 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbd3924-8c7a-4781-866c-edd55f3ac857 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:55.857373 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbd3924-8c7a-4781-866c-edd55f3ac857 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:55.872241 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-9fbd3924-8c7a-4781-866c-edd55f3ac857 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 01:00:55.872241 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:55.872241 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 01:00:55.872241 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 01:00:55.872241 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 01:00:55.872241 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 01:00:55.872241 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 01:00:55.872241 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 01:00:55.872241 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 01:00:55.872241 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:55.872241 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 01:00:55.872241 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:55.872241 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:55.872241 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:55.872241 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:55.872241 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:55.872241 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:55.873280 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:55.873280 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:55.873280 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:55.873280 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:55.873280 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:55.873280 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:55.873280 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 01:00:55.873280 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 01:00:55.873280 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 01:00:55.873280 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 01:00:55.873280 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:55.873280 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:55.873280 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:55.873280 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:55.873280 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:55.873280 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:55.874113 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:55.874113 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:55.874113 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:55.874113 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:55.874113 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:55.874113 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:55.874113 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:55.874113 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:55.874113 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:55.874113 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:55.874113 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 01:00:55.874113 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 01:00:55.874113 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 01:00:55.874113 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:55.880482 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 373/770] 10.222.0.42 () {56 vars in 1129 bytes} [Thu Sep 8 01:00:55 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:55.881359 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbd3924-8c7a-4781-866c-edd55f3ac857 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:55.915480 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 398/771] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:00:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 01:00:55.960846 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf0afd5-7296-470e-87d2-d2241b32c2fe None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:55.961164 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf0afd5-7296-470e-87d2-d2241b32c2fe None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:55.961382 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf0afd5-7296-470e-87d2-d2241b32c2fe None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:55.962846 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 399/772] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:00:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 01:00:55.969405 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b74b09-8f3a-412e-85e3-b33c109f3e6c None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:55.969683 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b74b09-8f3a-412e-85e3-b33c109f3e6c None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:55.969953 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b74b09-8f3a-412e-85e3-b33c109f3e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:55.999323 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-09b74b09-8f3a-412e-85e3-b33c109f3e6c None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:00:56.000535 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-09b74b09-8f3a-412e-85e3-b33c109f3e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:00:56.062082 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbd3924-8c7a-4781-866c-edd55f3ac857 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:56.129183 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 400/773] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:00:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 01:00:56.132175 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55288f7e-0bf6-457c-bcb8-63cbc7334427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:56.132790 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55288f7e-0bf6-457c-bcb8-63cbc7334427 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:56.133064 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55288f7e-0bf6-457c-bcb8-63cbc7334427 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:56.133331 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55288f7e-0bf6-457c-bcb8-63cbc7334427 None admin] PATH_INFO: `/v3/services` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:56.134080 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55288f7e-0bf6-457c-bcb8-63cbc7334427 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:56.134322 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55288f7e-0bf6-457c-bcb8-63cbc7334427 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:56.135785 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55288f7e-0bf6-457c-bcb8-63cbc7334427 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:56.141470 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 374/774] 10.222.0.42 () {56 vars in 1124 bytes} [Thu Sep 8 01:00:55 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:56.149327 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b74b09-8f3a-412e-85e3-b33c109f3e6c None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:56.187667 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:56.229293 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55288f7e-0bf6-457c-bcb8-63cbc7334427 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:56.265479 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b74b09-8f3a-412e-85e3-b33c109f3e6c None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:56.283510 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:56.303972 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495add34-2101-499a-9d57-060bb800b75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:56.304879 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495add34-2101-499a-9d57-060bb800b75e None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:56.305302 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495add34-2101-499a-9d57-060bb800b75e None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:56.305814 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495add34-2101-499a-9d57-060bb800b75e None admin] PATH_INFO: `/v3/services` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:56.306581 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495add34-2101-499a-9d57-060bb800b75e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:56.307049 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495add34-2101-499a-9d57-060bb800b75e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:56.309203 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495add34-2101-499a-9d57-060bb800b75e None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:56.314486 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 401/775] 10.222.0.42 () {56 vars in 1124 bytes} [Thu Sep 8 01:00:56 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:00:56.321998 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495add34-2101-499a-9d57-060bb800b75e None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:56.433952 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495add34-2101-499a-9d57-060bb800b75e None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:56.465050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:56.476852 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0effee0-6128-4d3c-b2d1-8518c9352116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:56.477720 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0effee0-6128-4d3c-b2d1-8518c9352116 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:56.478203 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0effee0-6128-4d3c-b2d1-8518c9352116 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:56.478677 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0effee0-6128-4d3c-b2d1-8518c9352116 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:56.479889 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0effee0-6128-4d3c-b2d1-8518c9352116 None admin] RBAC: Adding query filter params (interface=public, service_id=17aed1e7bf204e6db4a40d0268120f65, region_id=RegionOne) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:56.480483 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0effee0-6128-4d3c-b2d1-8518c9352116 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:56.482128 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0effee0-6128-4d3c-b2d1-8518c9352116 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:56.490092 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55288f7e-0bf6-457c-bcb8-63cbc7334427 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:56.490711 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 402/776] 10.222.0.42 () {56 vars in 1251 bytes} [Thu Sep 8 01:00:56 2022] GET /identity/v3/endpoints?service_id=17aed1e7bf204e6db4a40d0268120f65&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:56.497378 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0effee0-6128-4d3c-b2d1-8518c9352116 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:56.556940 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891eb578-17c4-467d-a519-3229cf3397f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:56.557576 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891eb578-17c4-467d-a519-3229cf3397f3 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:56.558366 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891eb578-17c4-467d-a519-3229cf3397f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:56.558793 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891eb578-17c4-467d-a519-3229cf3397f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:56.569780 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891eb578-17c4-467d-a519-3229cf3397f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:00:56.574897 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-891eb578-17c4-467d-a519-3229cf3397f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 01:00:56.574897 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:56.574897 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:56.574897 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:56.574897 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:56.574897 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:56.574897 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:56.574897 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:56.574897 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:56.574897 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:56.574897 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:56.574897 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:56.574897 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:00:56.574897 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:00:56.574897 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:00:56.574897 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:00:56.575846 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:00:56.575846 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:00:56.575846 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:00:56.575846 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:00:56.575846 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:56.575846 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:56.575846 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:56.575846 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:56.575846 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:56.575846 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:56.575846 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:56.575846 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:56.575846 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:56.575846 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:56.575846 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:56.575846 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:56.576734 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:56.576734 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:56.576734 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:56.576734 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:56.576734 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:00:56.576734 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:00:56.576734 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:00:56.576734 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:00:56.576734 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:00:56.576734 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:00:56.576734 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 01:00:56.576734 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:56.576734 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 375/777] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 8 01:00:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 01:00:56.577435 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891eb578-17c4-467d-a519-3229cf3397f3 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:56.633946 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0effee0-6128-4d3c-b2d1-8518c9352116 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:56.669755 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:56.676128 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d2ea5-ff04-496c-9bd7-798be6e538ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:56.677084 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d2ea5-ff04-496c-9bd7-798be6e538ee None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:56.677467 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d2ea5-ff04-496c-9bd7-798be6e538ee None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:56.677888 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d2ea5-ff04-496c-9bd7-798be6e538ee None admin] PATH_INFO: `/v3/services` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:56.678619 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330d2ea5-ff04-496c-9bd7-798be6e538ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:56.680303 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330d2ea5-ff04-496c-9bd7-798be6e538ee None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:56.686346 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891eb578-17c4-467d-a519-3229cf3397f3 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:56.687079 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 403/778] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Sep 8 01:00:56 2022] GET /identity/v3/services => generated 772 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:56.688068 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d2ea5-ff04-496c-9bd7-798be6e538ee None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:56.721043 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:56.776378 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2496a6-4172-49bc-8b98-ad16a4b3190a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:56.776944 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2496a6-4172-49bc-8b98-ad16a4b3190a None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:56.777224 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2496a6-4172-49bc-8b98-ad16a4b3190a None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:56.777498 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2496a6-4172-49bc-8b98-ad16a4b3190a None admin] PATH_INFO: `/v3/services/network` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:56.778304 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2496a6-4172-49bc-8b98-ad16a4b3190a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:56.779759 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2496a6-4172-49bc-8b98-ad16a4b3190a None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:56.806003 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-0f2496a6-4172-49bc-8b98-ad16a4b3190a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 01:00:56.806003 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:56.806003 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 01:00:56.806003 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 01:00:56.806003 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 01:00:56.806003 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 01:00:56.806003 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 01:00:56.806003 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 01:00:56.806003 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 01:00:56.806003 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:56.806003 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 01:00:56.806003 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:56.806003 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:56.806003 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:56.806003 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:56.806003 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:56.806003 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:56.808755 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:56.808755 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:56.808755 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:56.808755 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:56.808755 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:56.808755 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:56.808755 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 01:00:56.808755 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 01:00:56.808755 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 01:00:56.808755 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 01:00:56.808755 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:56.808755 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:56.808755 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:56.808755 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:56.808755 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:56.808755 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:56.810828 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:56.810828 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:56.810828 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:56.810828 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:56.810828 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:56.810828 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:56.810828 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:56.810828 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:56.810828 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:56.810828 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:56.810828 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 01:00:56.810828 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 01:00:56.810828 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 01:00:56.810828 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:56.810828 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 376/779] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 01:00:56 2022] GET /identity/v3/services/network => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:56.812967 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d2ea5-ff04-496c-9bd7-798be6e538ee None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:56.820163 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2496a6-4172-49bc-8b98-ad16a4b3190a None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:56.851631 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb24a89-984b-4aad-9d65-aeb08a6992c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:56.852303 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb24a89-984b-4aad-9d65-aeb08a6992c0 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:56.852589 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb24a89-984b-4aad-9d65-aeb08a6992c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:56.852915 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb24a89-984b-4aad-9d65-aeb08a6992c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:56.853715 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb24a89-984b-4aad-9d65-aeb08a6992c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:56.854032 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb24a89-984b-4aad-9d65-aeb08a6992c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:56.855617 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb24a89-984b-4aad-9d65-aeb08a6992c0 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:56.869962 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 404/780] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 01:00:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:56.876334 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb24a89-984b-4aad-9d65-aeb08a6992c0 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:56.974234 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2496a6-4172-49bc-8b98-ad16a4b3190a None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:56.999316 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:57.015136 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8be30ab-64de-4571-a50f-f0136a08a8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:57.015805 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be30ab-64de-4571-a50f-f0136a08a8f7 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:57.016086 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be30ab-64de-4571-a50f-f0136a08a8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:57.016758 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be30ab-64de-4571-a50f-f0136a08a8f7 None admin] PATH_INFO: `/v3/services` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:57.017079 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8be30ab-64de-4571-a50f-f0136a08a8f7 None admin] RBAC: Adding query filter params (name=network) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:57.017728 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8be30ab-64de-4571-a50f-f0136a08a8f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:57.018762 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8be30ab-64de-4571-a50f-f0136a08a8f7 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:57.020352 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb24a89-984b-4aad-9d65-aeb08a6992c0 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:57.024810 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 377/781] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Sep 8 01:00:56 2022] GET /identity/v3/services?name=network => generated 124 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:57.030950 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8be30ab-64de-4571-a50f-f0136a08a8f7 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:57.066426 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080eef17-84b4-4e4c-b5a0-e92f8891eeaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:57.067064 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:57.067565 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080eef17-84b4-4e4c-b5a0-e92f8891eeaa None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:57.067565 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080eef17-84b4-4e4c-b5a0-e92f8891eeaa None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:57.067852 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080eef17-84b4-4e4c-b5a0-e92f8891eeaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:57.091509 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080eef17-84b4-4e4c-b5a0-e92f8891eeaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:57.093035 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080eef17-84b4-4e4c-b5a0-e92f8891eeaa None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:57.123689 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-080eef17-84b4-4e4c-b5a0-e92f8891eeaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:57.123689 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:57.123689 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:57.123689 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:57.123689 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:57.123689 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:57.123689 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:57.123689 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:57.123689 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:57.123689 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:57.123689 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:57.123689 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:57.123689 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:00:57.123689 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:00:57.123689 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:00:57.123689 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:00:57.127775 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:57.127775 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:57.127775 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:57.127775 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:57.127775 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:57.127775 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:57.127775 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:57.127775 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:57.127775 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:57.127775 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:57.127775 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:57.127775 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:57.127775 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:57.127775 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:57.127775 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:57.127775 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:57.127775 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:00:57.129988 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:00:57.129988 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:57.129988 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:57.130380 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 405/782] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:00:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:57.134545 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080eef17-84b4-4e4c-b5a0-e92f8891eeaa None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:57.262762 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8be30ab-64de-4571-a50f-f0136a08a8f7 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:57.290179 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080eef17-84b4-4e4c-b5a0-e92f8891eeaa None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:57.306189 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc382e4c-a79b-417a-a658-0e6174df0397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:57.307250 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc382e4c-a79b-417a-a658-0e6174df0397 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:57.307682 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc382e4c-a79b-417a-a658-0e6174df0397 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:57.308660 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc382e4c-a79b-417a-a658-0e6174df0397 None admin] PATH_INFO: `/v3/services` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:57.309497 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc382e4c-a79b-417a-a658-0e6174df0397 None admin] RBAC: Adding query filter params (type=network) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:57.309971 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc382e4c-a79b-417a-a658-0e6174df0397 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:57.311678 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc382e4c-a79b-417a-a658-0e6174df0397 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:57.318368 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 378/783] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Sep 8 01:00:57 2022] GET /identity/v3/services?type=network => generated 350 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:57.321391 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c8bff-47ce-46cd-a28b-92560d58f375 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:57.321670 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c8bff-47ce-46cd-a28b-92560d58f375 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:57.321938 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c8bff-47ce-46cd-a28b-92560d58f375 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:57.323348 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 379/784] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:00:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:00:57.324867 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52c8bff-47ce-46cd-a28b-92560d58f375 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:57.341792 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5ce15d-e0d0-4813-9a6d-4ebd031336e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:57.342466 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ce15d-e0d0-4813-9a6d-4ebd031336e9 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:57.343149 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ce15d-e0d0-4813-9a6d-4ebd031336e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:57.343442 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ce15d-e0d0-4813-9a6d-4ebd031336e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:57.344495 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5ce15d-e0d0-4813-9a6d-4ebd031336e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:57.344805 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5ce15d-e0d0-4813-9a6d-4ebd031336e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:57.346456 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5ce15d-e0d0-4813-9a6d-4ebd031336e9 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:57.349545 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:57.362619 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 406/785] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:00:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:57.363985 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a51e0-2a51-434c-9d6e-b5deed7ddc3a None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:57.364382 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a51e0-2a51-434c-9d6e-b5deed7ddc3a None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:57.364716 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a51e0-2a51-434c-9d6e-b5deed7ddc3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:57.387940 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-627a51e0-2a51-434c-9d6e-b5deed7ddc3a None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:00:57.389896 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-627a51e0-2a51-434c-9d6e-b5deed7ddc3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:00:57.427528 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:57.453147 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52c8bff-47ce-46cd-a28b-92560d58f375 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:57.480683 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 407/786] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:00:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 01:00:57.482096 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627a51e0-2a51-434c-9d6e-b5deed7ddc3a None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:57.492726 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b39566-b8d0-4071-8f86-f01680390f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:57.493338 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b39566-b8d0-4071-8f86-f01680390f1c None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:57.493674 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b39566-b8d0-4071-8f86-f01680390f1c None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:57.493950 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b39566-b8d0-4071-8f86-f01680390f1c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:57.494714 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b39566-b8d0-4071-8f86-f01680390f1c None admin] RBAC: Adding query filter params (interface=public, service_id=b544884d09aa476c837e7aceddc8bb00, region_id=RegionOne) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:57.495024 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b39566-b8d0-4071-8f86-f01680390f1c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:57.497120 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b39566-b8d0-4071-8f86-f01680390f1c None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:57.505665 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 380/787] 10.222.0.42 () {56 vars in 1251 bytes} [Thu Sep 8 01:00:57 2022] GET /identity/v3/endpoints?service_id=b544884d09aa476c837e7aceddc8bb00&interface=public®ion_id=RegionOne => generated 194 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:57.514787 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b39566-b8d0-4071-8f86-f01680390f1c None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:57.607940 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627a51e0-2a51-434c-9d6e-b5deed7ddc3a None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:57.638580 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:57.656268 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaca219-bc53-4844-9dba-55dd1b6385cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:57.656914 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaca219-bc53-4844-9dba-55dd1b6385cb None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:57.657194 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaca219-bc53-4844-9dba-55dd1b6385cb None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:57.657468 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaca219-bc53-4844-9dba-55dd1b6385cb None admin] PATH_INFO: `/v3/users/swift` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:57.674017 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaca219-bc53-4844-9dba-55dd1b6385cb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:57.675482 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaca219-bc53-4844-9dba-55dd1b6385cb None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:57.698379 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-0eaca219-bc53-4844-9dba-55dd1b6385cb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 08 01:00:57.698379 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:57.698379 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:57.698379 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:57.698379 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:57.698379 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:57.698379 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:57.698379 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:57.698379 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:57.698379 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:57.698379 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:57.698379 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:57.698379 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 01:00:57.698379 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 01:00:57.698379 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 01:00:57.698379 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 01:00:57.700079 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:57.700079 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:57.700079 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 01:00:57.700079 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:00:57.700079 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 01:00:57.700079 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:00:57.700079 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:57.700079 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:57.700079 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:57.700079 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:57.700079 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:57.700079 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:57.700079 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:57.700079 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:57.700079 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:57.700079 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:57.700079 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:57.700079 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:57.701087 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:57.701087 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:57.701087 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 01:00:57.701087 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 01:00:57.701087 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 01:00:57.701087 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 01:00:57.701087 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 01:00:57.701087 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 01:00:57.701087 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 08 01:00:57.701087 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:57.707946 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 408/788] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 8 01:00:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:57.713415 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaca219-bc53-4844-9dba-55dd1b6385cb None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:57.726485 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b39566-b8d0-4071-8f86-f01680390f1c None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:57.798735 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45db4ce-646f-4716-bd17-2ecd0947dff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:57.799270 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45db4ce-646f-4716-bd17-2ecd0947dff9 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:57.799546 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45db4ce-646f-4716-bd17-2ecd0947dff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:57.799818 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45db4ce-646f-4716-bd17-2ecd0947dff9 None admin] PATH_INFO: `/v3/services` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:57.800458 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45db4ce-646f-4716-bd17-2ecd0947dff9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:57.801835 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45db4ce-646f-4716-bd17-2ecd0947dff9 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:57.814422 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 381/789] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Sep 8 01:00:57 2022] GET /identity/v3/services => generated 772 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:57.829317 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e7bd07-fe6f-4696-9bad-5f915f34c2e8 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:57.829660 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e7bd07-fe6f-4696-9bad-5f915f34c2e8 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:57.829861 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e7bd07-fe6f-4696-9bad-5f915f34c2e8 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:57.831712 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 382/790] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:00:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 01:00:57.837620 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35bb175-be86-4508-9c4d-770780b79d96 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:57.837945 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35bb175-be86-4508-9c4d-770780b79d96 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:57.838187 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35bb175-be86-4508-9c4d-770780b79d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:57.855740 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:57.869584 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-d35bb175-be86-4508-9c4d-770780b79d96 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:00:57.871624 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-d35bb175-be86-4508-9c4d-770780b79d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:00:57.911663 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaca219-bc53-4844-9dba-55dd1b6385cb None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:57.958360 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f220ea-45af-4f1b-9072-1111bd9cb709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:57.959496 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f220ea-45af-4f1b-9072-1111bd9cb709 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:57.960022 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f220ea-45af-4f1b-9072-1111bd9cb709 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:57.960518 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f220ea-45af-4f1b-9072-1111bd9cb709 None admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:57.961483 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f220ea-45af-4f1b-9072-1111bd9cb709 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:57.962036 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f220ea-45af-4f1b-9072-1111bd9cb709 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:57.963874 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f220ea-45af-4f1b-9072-1111bd9cb709 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:57.980613 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 383/791] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:00:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 01:00:57.982131 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35bb175-be86-4508-9c4d-770780b79d96 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:57.991324 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 409/792] 10.222.0.42 () {56 vars in 1135 bytes} [Thu Sep 8 01:00:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:57.993895 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f220ea-45af-4f1b-9072-1111bd9cb709 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:58.058397 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:58.133018 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f220ea-45af-4f1b-9072-1111bd9cb709 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:58.150350 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35bb175-be86-4508-9c4d-770780b79d96 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:58.201737 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e5c785-6d3e-46e1-9c96-f49731ca46d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:58.202374 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5c785-6d3e-46e1-9c96-f49731ca46d8 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:58.202655 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5c785-6d3e-46e1-9c96-f49731ca46d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:58.202926 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5c785-6d3e-46e1-9c96-f49731ca46d8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:58.203970 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12c8f8b-dc37-4ce4-ad96-05955af53852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:58.204621 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c8f8b-dc37-4ce4-ad96-05955af53852 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:58.204895 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c8f8b-dc37-4ce4-ad96-05955af53852 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:58.205163 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c8f8b-dc37-4ce4-ad96-05955af53852 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:58.209411 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:58.223604 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12c8f8b-dc37-4ce4-ad96-05955af53852 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:58.225017 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e5c785-6d3e-46e1-9c96-f49731ca46d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:00:58.225317 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12c8f8b-dc37-4ce4-ad96-05955af53852 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:58.230367 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-26e5c785-6d3e-46e1-9c96-f49731ca46d8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 01:00:58.230367 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:58.230367 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:58.230367 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:58.230367 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:58.230367 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:58.230367 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:58.230367 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:58.230367 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:58.230367 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:58.230367 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:58.230367 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:58.230367 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:00:58.230367 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:00:58.230367 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:00:58.230367 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:00:58.231303 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:00:58.231303 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:00:58.231303 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:00:58.231303 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:00:58.231303 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:58.231303 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:58.231303 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:58.231303 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:58.231303 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:58.231303 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:58.231303 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:58.231303 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:58.231303 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:58.231303 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:58.231303 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:58.231303 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:58.232187 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:58.232187 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:58.232187 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:58.232187 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:58.232187 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:00:58.232187 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:00:58.232187 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:00:58.232187 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:00:58.232187 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:00:58.232187 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:00:58.232187 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 01:00:58.232187 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:58.240857 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 384/793] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 8 01:00:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:58.240993 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-f12c8f8b-dc37-4ce4-ad96-05955af53852 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:58.240993 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:58.240993 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:58.240993 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:58.240993 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:58.240993 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:58.240993 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:58.240993 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:58.240993 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:58.240993 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:58.240993 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:58.240993 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:58.240993 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:00:58.240993 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:00:58.240993 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:00:58.240993 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:00:58.241876 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:58.241876 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:58.241876 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:58.241876 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:58.241876 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:58.241876 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:58.241876 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:58.241876 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:58.241876 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:58.241876 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:58.241876 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:58.241876 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:58.241876 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:58.241876 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:58.241876 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:58.241876 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:58.241876 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:00:58.242759 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:00:58.242759 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:58.242759 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:58.242759 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 410/794] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:00:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:58.243400 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12c8f8b-dc37-4ce4-ad96-05955af53852 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:58.248454 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e5c785-6d3e-46e1-9c96-f49731ca46d8 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:58.357664 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:58.367021 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12c8f8b-dc37-4ce4-ad96-05955af53852 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:58.404954 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad732da-5f53-44a8-9a7e-8600d2e0ca42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:58.405998 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad732da-5f53-44a8-9a7e-8600d2e0ca42 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:58.406274 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad732da-5f53-44a8-9a7e-8600d2e0ca42 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:58.406809 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad732da-5f53-44a8-9a7e-8600d2e0ca42 None admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:58.407485 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad732da-5f53-44a8-9a7e-8600d2e0ca42 None admin] RBAC: Adding query filter params (name=service) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:58.407785 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad732da-5f53-44a8-9a7e-8600d2e0ca42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:58.409214 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad732da-5f53-44a8-9a7e-8600d2e0ca42 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:58.419688 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 411/795] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 8 01:00:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:58.419832 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e5c785-6d3e-46e1-9c96-f49731ca46d8 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:58.420532 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad732da-5f53-44a8-9a7e-8600d2e0ca42 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:58.468310 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c499ae46-9137-4f83-9899-b0f03d396995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:58.468926 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499ae46-9137-4f83-9899-b0f03d396995 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:58.469205 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499ae46-9137-4f83-9899-b0f03d396995 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:58.469475 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499ae46-9137-4f83-9899-b0f03d396995 None admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:58.470124 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c499ae46-9137-4f83-9899-b0f03d396995 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:58.470426 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c499ae46-9137-4f83-9899-b0f03d396995 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:58.471948 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c499ae46-9137-4f83-9899-b0f03d396995 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:58.485053 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 385/796] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:00:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:00:58.485698 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c499ae46-9137-4f83-9899-b0f03d396995 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:58.556565 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:58.595809 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c499ae46-9137-4f83-9899-b0f03d396995 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:58.645113 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c403b7-c46b-431a-a63e-fa677a710783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:58.646007 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c403b7-c46b-431a-a63e-fa677a710783 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:58.646434 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c403b7-c46b-431a-a63e-fa677a710783 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:58.646850 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c403b7-c46b-431a-a63e-fa677a710783 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:58.659862 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c403b7-c46b-431a-a63e-fa677a710783 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:58.661599 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c403b7-c46b-431a-a63e-fa677a710783 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:58.680630 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-d1c403b7-c46b-431a-a63e-fa677a710783 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:58.680630 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:58.680630 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:58.680630 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:58.680630 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:58.680630 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:58.680630 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:58.680630 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:58.680630 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:58.680630 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:58.680630 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:58.680630 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:58.680630 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:00:58.680630 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:00:58.680630 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:00:58.680630 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:00:58.681807 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:58.681807 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:58.681807 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:58.681807 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:58.681807 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:58.681807 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:58.681807 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:58.681807 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:58.681807 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:58.681807 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:58.681807 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:58.681807 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:58.681807 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:58.681807 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:58.681807 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:58.681807 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:58.681807 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:00:58.682792 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:00:58.682792 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:58.682792 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:58.686031 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 386/797] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:00:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:58.686970 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c403b7-c46b-431a-a63e-fa677a710783 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:58.697853 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad732da-5f53-44a8-9a7e-8600d2e0ca42 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:58.737425 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc34f86d-c07d-4d5a-b418-f64f5c5f41a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:58.738334 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc34f86d-c07d-4d5a-b418-f64f5c5f41a0 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:58.738826 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc34f86d-c07d-4d5a-b418-f64f5c5f41a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:58.739243 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc34f86d-c07d-4d5a-b418-f64f5c5f41a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:58.751386 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc34f86d-c07d-4d5a-b418-f64f5c5f41a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:00:58.756853 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-cc34f86d-c07d-4d5a-b418-f64f5c5f41a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 01:00:58.756853 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:58.756853 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:58.756853 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:58.756853 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:58.756853 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:58.756853 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:58.756853 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:58.756853 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:58.756853 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:58.756853 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:58.756853 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:58.756853 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:00:58.756853 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:00:58.756853 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:00:58.756853 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:00:58.757967 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:00:58.757967 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:00:58.757967 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:00:58.757967 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:00:58.757967 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:58.757967 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:58.757967 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:58.757967 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:58.757967 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:58.757967 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:58.757967 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:58.757967 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:58.757967 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:58.757967 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:58.757967 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:58.757967 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:58.758806 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:58.758806 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:58.758806 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:58.758806 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:58.758806 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:00:58.758806 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:00:58.758806 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:00:58.758806 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:00:58.758806 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:00:58.758806 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:00:58.758806 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 01:00:58.758806 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:58.761325 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 412/798] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 8 01:00:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:58.762936 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc34f86d-c07d-4d5a-b418-f64f5c5f41a0 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:58.789094 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:58.811149 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c403b7-c46b-431a-a63e-fa677a710783 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:58.821689 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:58.853957 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6332e1ce-7f70-479c-acf2-7cbdfbbe7038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:58.854589 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6332e1ce-7f70-479c-acf2-7cbdfbbe7038 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:58.855083 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6332e1ce-7f70-479c-acf2-7cbdfbbe7038 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:58.855465 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6332e1ce-7f70-479c-acf2-7cbdfbbe7038 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:58.864276 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6332e1ce-7f70-479c-acf2-7cbdfbbe7038 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:58.866015 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6332e1ce-7f70-479c-acf2-7cbdfbbe7038 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:58.881867 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-6332e1ce-7f70-479c-acf2-7cbdfbbe7038 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:00:58.881867 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:58.881867 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:58.881867 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:58.881867 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:58.881867 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:58.881867 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:58.881867 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:58.881867 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:58.881867 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:58.881867 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:58.881867 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:58.881867 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 01:00:58.881867 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 01:00:58.881867 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 01:00:58.882972 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 01:00:58.882972 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:58.882972 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:58.882972 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:58.882972 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:58.882972 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:58.882972 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:58.882972 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:58.882972 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:58.882972 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:58.882972 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:58.882972 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:58.882972 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:58.882972 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:58.882972 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:58.882972 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:58.882972 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:58.883891 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 01:00:58.883891 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 01:00:58.883891 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 01:00:58.883891 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 01:00:58.883891 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 01:00:58.883891 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 01:00:58.883891 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:00:58.883891 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:58.883891 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 387/799] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 01:00:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:58.884934 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6332e1ce-7f70-479c-acf2-7cbdfbbe7038 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:58.889306 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc34f86d-c07d-4d5a-b418-f64f5c5f41a0 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:58.928590 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c391efa5-6751-4330-a48a-fd4f3552d428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:58.929760 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391efa5-6751-4330-a48a-fd4f3552d428 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:58.930265 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391efa5-6751-4330-a48a-fd4f3552d428 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:58.930732 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391efa5-6751-4330-a48a-fd4f3552d428 None admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:58.931625 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c391efa5-6751-4330-a48a-fd4f3552d428 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:58.932147 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c391efa5-6751-4330-a48a-fd4f3552d428 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:58.934464 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c391efa5-6751-4330-a48a-fd4f3552d428 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:58.951809 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 413/800] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:00:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:58.953011 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c391efa5-6751-4330-a48a-fd4f3552d428 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:59.021932 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6332e1ce-7f70-479c-acf2-7cbdfbbe7038 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:59.049028 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:59.071924 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80609b1-53a5-40b8-9e17-44724540b7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:59.072557 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80609b1-53a5-40b8-9e17-44724540b7dd None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:59.072836 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80609b1-53a5-40b8-9e17-44724540b7dd None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:59.073105 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80609b1-53a5-40b8-9e17-44724540b7dd None admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:59.075040 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80609b1-53a5-40b8-9e17-44724540b7dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:59.075343 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80609b1-53a5-40b8-9e17-44724540b7dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:59.076756 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80609b1-53a5-40b8-9e17-44724540b7dd None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:59.082869 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c391efa5-6751-4330-a48a-fd4f3552d428 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:59.085758 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 388/801] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 01:00:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:59.087101 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80609b1-53a5-40b8-9e17-44724540b7dd None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:59.118345 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:59.122961 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223926ec-cfae-40f9-988a-b961d3650b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:59.123572 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223926ec-cfae-40f9-988a-b961d3650b2e None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:59.123855 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223926ec-cfae-40f9-988a-b961d3650b2e None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:59.124129 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223926ec-cfae-40f9-988a-b961d3650b2e None admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:59.125401 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223926ec-cfae-40f9-988a-b961d3650b2e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:59.125401 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223926ec-cfae-40f9-988a-b961d3650b2e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:59.126583 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223926ec-cfae-40f9-988a-b961d3650b2e None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:59.139945 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 414/802] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 8 01:00:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:59.140835 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223926ec-cfae-40f9-988a-b961d3650b2e None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:59.197809 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80609b1-53a5-40b8-9e17-44724540b7dd None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:59.236698 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2146f51-2173-4877-8b74-6dfc531bfa92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:59.237338 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2146f51-2173-4877-8b74-6dfc531bfa92 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:59.237625 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2146f51-2173-4877-8b74-6dfc531bfa92 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:59.237898 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2146f51-2173-4877-8b74-6dfc531bfa92 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:59.246369 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2146f51-2173-4877-8b74-6dfc531bfa92 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:59.248525 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2146f51-2173-4877-8b74-6dfc531bfa92 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:59.252696 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223926ec-cfae-40f9-988a-b961d3650b2e None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:59.260362 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-d2146f51-2173-4877-8b74-6dfc531bfa92 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 01:00:59.260362 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:59.260362 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:59.260362 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:59.260362 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:59.260362 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:59.260362 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:59.260362 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:59.260362 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:59.260362 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:59.260362 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:59.260362 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:59.260362 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 01:00:59.260362 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 01:00:59.260362 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 01:00:59.260362 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 01:00:59.261867 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:59.261867 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:59.261867 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 01:00:59.261867 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:00:59.261867 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 01:00:59.261867 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:00:59.261867 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:59.261867 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:59.261867 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:59.261867 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:59.261867 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:59.261867 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:59.261867 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:59.261867 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:59.261867 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:59.261867 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:59.261867 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:59.261867 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:59.262827 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:59.262827 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:59.262827 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 01:00:59.262827 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 01:00:59.262827 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 01:00:59.262827 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 01:00:59.262827 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 01:00:59.262827 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 01:00:59.262827 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 01:00:59.262827 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:59.262827 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 389/803] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Sep 8 01:00:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:59.262827 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d5fc5-958a-43c8-b6e1-eb44888b3838 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:59.263480 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d5fc5-958a-43c8-b6e1-eb44888b3838 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:59.263480 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d5fc5-958a-43c8-b6e1-eb44888b3838 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:59.264674 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 390/804] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:00:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:00:59.265202 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8d5fc5-958a-43c8-b6e1-eb44888b3838 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:59.294983 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942a245f-08b3-4e5a-9589-e6adb7b5b3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:59.295868 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a245f-08b3-4e5a-9589-e6adb7b5b3a3 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:59.296292 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a245f-08b3-4e5a-9589-e6adb7b5b3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:59.296573 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a245f-08b3-4e5a-9589-e6adb7b5b3a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:59.305563 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:59.310491 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942a245f-08b3-4e5a-9589-e6adb7b5b3a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:59.311947 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942a245f-08b3-4e5a-9589-e6adb7b5b3a3 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:59.326762 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-942a245f-08b3-4e5a-9589-e6adb7b5b3a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:59.326762 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:59.326762 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:59.326762 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:59.326762 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:59.326762 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:59.326762 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:59.326762 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:59.326762 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:59.326762 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:59.326762 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:59.326762 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:59.326762 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:00:59.326762 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:00:59.326762 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:00:59.326762 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:00:59.327701 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:59.327701 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:59.327701 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:59.327701 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:59.327701 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:59.327701 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:59.327701 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:59.327701 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:59.327701 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:59.327701 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:59.327701 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:59.327701 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:59.327701 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:59.327701 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:59.327701 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:59.327701 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:59.327701 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:00:59.328638 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:00:59.328638 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:59.328638 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:59.328638 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 415/805] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:00:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:59.328638 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942a245f-08b3-4e5a-9589-e6adb7b5b3a3 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:59.372575 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:59.384647 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8d5fc5-958a-43c8-b6e1-eb44888b3838 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:59.424629 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e63bfa-3754-4f82-a523-725ab1012f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:59.425220 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e63bfa-3754-4f82-a523-725ab1012f4d None admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:59.425502 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e63bfa-3754-4f82-a523-725ab1012f4d None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:59.425771 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e63bfa-3754-4f82-a523-725ab1012f4d None admin] PATH_INFO: `/v3/projects/78ad2a8fb5264fb4af935e039d28c8de/users/8add26872fe6479aa9d7de155a17195a/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:59.427688 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e63bfa-3754-4f82-a523-725ab1012f4d None admin] RBAC: Authorizing `identity:create_grant(project_id=78ad2a8fb5264fb4af935e039d28c8de, user_id=8add26872fe6479aa9d7de155a17195a, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:59.429228 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e63bfa-3754-4f82-a523-725ab1012f4d None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:59.439116 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942a245f-08b3-4e5a-9589-e6adb7b5b3a3 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:59.456188 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 391/806] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 8 01:00:59 2022] PUT /identity/v3/projects/78ad2a8fb5264fb4af935e039d28c8de/users/8add26872fe6479aa9d7de155a17195a/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:00:59.456188 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e0046d-3544-4634-a2cf-b7cc865cfd2e None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:59.456188 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e0046d-3544-4634-a2cf-b7cc865cfd2e None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:59.456188 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e0046d-3544-4634-a2cf-b7cc865cfd2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:59.468350 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-39e0046d-3544-4634-a2cf-b7cc865cfd2e None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:00:59.469788 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-39e0046d-3544-4634-a2cf-b7cc865cfd2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:00:59.491758 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba0e13-a296-4de4-bc02-89ed24ee7fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:59.492406 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba0e13-a296-4de4-bc02-89ed24ee7fa0 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:59.492686 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba0e13-a296-4de4-bc02-89ed24ee7fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:59.492958 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba0e13-a296-4de4-bc02-89ed24ee7fa0 None admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:59.493701 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ba0e13-a296-4de4-bc02-89ed24ee7fa0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:59.494121 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ba0e13-a296-4de4-bc02-89ed24ee7fa0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:59.495707 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ba0e13-a296-4de4-bc02-89ed24ee7fa0 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:59.524072 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 416/807] 10.222.0.42 () {56 vars in 1149 bytes} [Thu Sep 8 01:00:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:59.525681 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba0e13-a296-4de4-bc02-89ed24ee7fa0 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:59.552050 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 392/808] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:00:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 01:00:59.553076 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e0046d-3544-4634-a2cf-b7cc865cfd2e None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:59.626356 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:59.671562 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:59.698746 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e0046d-3544-4634-a2cf-b7cc865cfd2e None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:59.712659 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba0e13-a296-4de4-bc02-89ed24ee7fa0 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:59.743279 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3781ef75-1dd5-4842-89ac-5b2b8da18873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:59.744328 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3781ef75-1dd5-4842-89ac-5b2b8da18873 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:59.745058 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3781ef75-1dd5-4842-89ac-5b2b8da18873 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:59.746282 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3781ef75-1dd5-4842-89ac-5b2b8da18873 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:59.757770 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2fb939-167b-422c-8db9-44d456695bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:59.758393 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2fb939-167b-422c-8db9-44d456695bdd None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:59.758697 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2fb939-167b-422c-8db9-44d456695bdd None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:59.759304 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2fb939-167b-422c-8db9-44d456695bdd None admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:59.759913 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3781ef75-1dd5-4842-89ac-5b2b8da18873 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:59.760055 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2fb939-167b-422c-8db9-44d456695bdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:00:59.760278 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2fb939-167b-422c-8db9-44d456695bdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:59.762219 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2fb939-167b-422c-8db9-44d456695bdd None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:59.762421 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3781ef75-1dd5-4842-89ac-5b2b8da18873 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:59.776626 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 417/809] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:00:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:00:59.780310 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-3781ef75-1dd5-4842-89ac-5b2b8da18873 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:59.780310 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:59.780310 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:59.780310 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:59.780310 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:59.780310 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:59.780310 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:59.780310 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:59.780310 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:59.780310 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:59.780310 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:59.780310 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:59.780310 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:00:59.780310 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:00:59.780310 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:00:59.780310 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:00:59.781484 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:59.781484 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:59.781484 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:59.781484 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:59.781484 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:59.781484 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:59.781484 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:59.781484 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:59.781484 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:59.781484 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:59.781484 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:59.781484 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:59.781484 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:59.781484 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:59.781484 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:59.781484 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:59.781484 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:00:59.782371 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:00:59.782371 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:00:59.782371 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:00:59.782538 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2fb939-167b-422c-8db9-44d456695bdd None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:59.784380 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 393/810] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:00:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:59.796743 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3781ef75-1dd5-4842-89ac-5b2b8da18873 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:59.922238 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2fb939-167b-422c-8db9-44d456695bdd None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:59.932979 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:00:59.956919 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3781ef75-1dd5-4842-89ac-5b2b8da18873 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:00:59.962580 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507cf74c-889d-4cae-ac30-4d2f4a912aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:00:59.963320 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507cf74c-889d-4cae-ac30-4d2f4a912aee None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:00:59.963674 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507cf74c-889d-4cae-ac30-4d2f4a912aee None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:00:59.964044 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507cf74c-889d-4cae-ac30-4d2f4a912aee None admin] PATH_INFO: `/v3/users/nova` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:00:59.972514 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507cf74c-889d-4cae-ac30-4d2f4a912aee None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:00:59.974746 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507cf74c-889d-4cae-ac30-4d2f4a912aee None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:00:59.988441 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-507cf74c-889d-4cae-ac30-4d2f4a912aee None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 01:00:59.988441 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:00:59.988441 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:00:59.988441 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:00:59.988441 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:00:59.988441 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:00:59.988441 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:00:59.988441 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:00:59.988441 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:00:59.988441 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:00:59.988441 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:00:59.988441 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:00:59.988441 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 01:00:59.988441 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 01:00:59.988441 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 01:00:59.988441 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 01:00:59.989418 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:00:59.989418 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:00:59.989418 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 01:00:59.989418 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:00:59.989418 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 01:00:59.989418 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:00:59.989418 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:00:59.989418 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:00:59.989418 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:00:59.989418 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:00:59.989418 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:00:59.989418 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:00:59.989418 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:00:59.989418 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:00:59.989418 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:00:59.989418 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:00:59.989418 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:00:59.989418 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:00:59.990386 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:00:59.990386 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:00:59.990386 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 01:00:59.990386 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 01:00:59.990386 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 01:00:59.990386 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 01:00:59.990386 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 01:00:59.990386 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 01:00:59.990386 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 01:00:59.990386 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:00:59.994485 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 418/811] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Sep 8 01:00:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:00:59.995353 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507cf74c-889d-4cae-ac30-4d2f4a912aee None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:00:59.995896 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:00.003498 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c61e481-7dbd-41cc-a060-5d7a821c2021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:00.004378 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c61e481-7dbd-41cc-a060-5d7a821c2021 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:00.004805 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c61e481-7dbd-41cc-a060-5d7a821c2021 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:00.005221 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c61e481-7dbd-41cc-a060-5d7a821c2021 None admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:00.006008 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c61e481-7dbd-41cc-a060-5d7a821c2021 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:00.006455 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c61e481-7dbd-41cc-a060-5d7a821c2021 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:00.008662 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c61e481-7dbd-41cc-a060-5d7a821c2021 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:00.026155 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 394/812] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:00:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:00.027383 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c61e481-7dbd-41cc-a060-5d7a821c2021 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:00.116698 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507cf74c-889d-4cae-ac30-4d2f4a912aee None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:00.160694 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1a3531-266d-4eda-ba8b-1d484ef9e3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:00.161345 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1a3531-266d-4eda-ba8b-1d484ef9e3bb None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:00.161628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1a3531-266d-4eda-ba8b-1d484ef9e3bb None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:00.161902 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1a3531-266d-4eda-ba8b-1d484ef9e3bb None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:00.162574 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1a3531-266d-4eda-ba8b-1d484ef9e3bb None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:00.164687 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1a3531-266d-4eda-ba8b-1d484ef9e3bb None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:00.167488 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c61e481-7dbd-41cc-a060-5d7a821c2021 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:00.178161 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-bf1a3531-266d-4eda-ba8b-1d484ef9e3bb None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 01:01:00.178161 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:00.178161 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 01:01:00.178161 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 01:01:00.178161 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 01:01:00.178161 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 01:01:00.178161 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 01:01:00.178161 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 01:01:00.178161 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 01:01:00.178161 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:00.178161 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 01:01:00.178161 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:00.178161 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:00.178161 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:00.178161 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:00.178161 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:00.178161 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:00.179188 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:00.179188 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:00.179188 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:00.179188 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:00.179188 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:00.179188 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:00.179188 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 01:01:00.179188 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 01:01:00.179188 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 01:01:00.179188 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 01:01:00.179188 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:00.179188 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:00.179188 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:00.179188 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:00.179188 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:00.179188 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:00.180204 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:00.180204 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:00.180204 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:00.180204 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:00.180204 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:00.180204 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:00.180204 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:00.180204 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:00.180204 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:00.180204 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:00.180204 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 01:01:00.180204 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 01:01:00.180204 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 01:01:00.180204 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:00.180204 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 419/813] 10.222.0.42 () {56 vars in 1129 bytes} [Thu Sep 8 01:00:59 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:01:00.181002 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1a3531-266d-4eda-ba8b-1d484ef9e3bb None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:00.211403 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822589f8-857b-43d6-8b5e-3ee488a78eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:00.212220 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822589f8-857b-43d6-8b5e-3ee488a78eca None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:00.212915 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822589f8-857b-43d6-8b5e-3ee488a78eca None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:00.213199 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822589f8-857b-43d6-8b5e-3ee488a78eca None admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:00.214732 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822589f8-857b-43d6-8b5e-3ee488a78eca None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:00.215045 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822589f8-857b-43d6-8b5e-3ee488a78eca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:00.216959 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:00.217904 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822589f8-857b-43d6-8b5e-3ee488a78eca None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:00.244227 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 395/814] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Sep 8 01:01:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:00.245833 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822589f8-857b-43d6-8b5e-3ee488a78eca None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:00.302119 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1a3531-266d-4eda-ba8b-1d484ef9e3bb None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:00.305909 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:00.342180 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d092b203-193b-48e1-953e-b91860023a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:00.343165 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d092b203-193b-48e1-953e-b91860023a40 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:00.343674 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d092b203-193b-48e1-953e-b91860023a40 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:00.344180 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d092b203-193b-48e1-953e-b91860023a40 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:00.351768 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d092b203-193b-48e1-953e-b91860023a40 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:00.353500 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d092b203-193b-48e1-953e-b91860023a40 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:00.358047 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822589f8-857b-43d6-8b5e-3ee488a78eca None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:00.366423 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-d092b203-193b-48e1-953e-b91860023a40 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:01:00.366423 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:00.366423 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:00.366423 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:00.366423 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:00.366423 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:00.366423 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:00.366423 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:00.366423 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:00.366423 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:00.366423 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:00.366423 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:00.366423 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 01:01:00.366423 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 01:01:00.366423 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 01:01:00.367302 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 01:01:00.367302 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:00.367302 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:00.367302 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:00.367302 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:00.367302 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:00.367302 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:00.367302 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:00.367302 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:00.367302 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:00.367302 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:00.367302 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:00.367302 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:00.367302 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:00.367302 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:00.367302 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:00.367302 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:00.368241 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 01:01:00.368241 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 01:01:00.368241 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 01:01:00.368241 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 01:01:00.368241 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 01:01:00.368241 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 01:01:00.368241 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:01:00.368241 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:00.375369 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 420/815] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 01:01:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:01:00.375369 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d092b203-193b-48e1-953e-b91860023a40 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:00.394587 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4ff72b-1fd6-493e-b3ea-932923ac35a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:00.395202 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4ff72b-1fd6-493e-b3ea-932923ac35a1 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:00.395482 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4ff72b-1fd6-493e-b3ea-932923ac35a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:00.395759 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4ff72b-1fd6-493e-b3ea-932923ac35a1 None admin] PATH_INFO: `/v3/services` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:00.396423 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4ff72b-1fd6-493e-b3ea-932923ac35a1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:00.396734 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4ff72b-1fd6-493e-b3ea-932923ac35a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:00.398111 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4ff72b-1fd6-493e-b3ea-932923ac35a1 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:00.403259 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 396/816] 10.222.0.42 () {56 vars in 1124 bytes} [Thu Sep 8 01:01:00 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:00.404438 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a7d79-bb40-4b02-bf6b-e02c18cfb1bc None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:00.404720 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a7d79-bb40-4b02-bf6b-e02c18cfb1bc None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:00.404996 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a7d79-bb40-4b02-bf6b-e02c18cfb1bc None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:00.408516 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 397/817] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:01:00.408516 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13a7d79-bb40-4b02-bf6b-e02c18cfb1bc None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:00.484116 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:00.493436 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d092b203-193b-48e1-953e-b91860023a40 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:00.520364 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13a7d79-bb40-4b02-bf6b-e02c18cfb1bc None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:00.534447 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d5c96e-e7dc-4644-9d72-1cbb7933a2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:00.535067 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d5c96e-e7dc-4644-9d72-1cbb7933a2a0 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:00.535348 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d5c96e-e7dc-4644-9d72-1cbb7933a2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:00.535622 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d5c96e-e7dc-4644-9d72-1cbb7933a2a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:00.537451 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:00.560013 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74071d2-20d1-4535-8a32-0d17a844c953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:00.560792 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74071d2-20d1-4535-8a32-0d17a844c953 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:00.561071 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74071d2-20d1-4535-8a32-0d17a844c953 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:00.561337 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74071d2-20d1-4535-8a32-0d17a844c953 None admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:00.562026 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74071d2-20d1-4535-8a32-0d17a844c953 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:00.562317 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74071d2-20d1-4535-8a32-0d17a844c953 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:00.563740 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74071d2-20d1-4535-8a32-0d17a844c953 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:00.576240 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 398/818] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 8 01:01:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:00.577393 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74071d2-20d1-4535-8a32-0d17a844c953 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:00.611148 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d5c96e-e7dc-4644-9d72-1cbb7933a2a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:00.613549 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d5c96e-e7dc-4644-9d72-1cbb7933a2a0 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:00.686884 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74071d2-20d1-4535-8a32-0d17a844c953 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:00.715426 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:00.722813 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ea3fd-13fe-4363-b4c4-880f87a1ab6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:00.723589 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ea3fd-13fe-4363-b4c4-880f87a1ab6e None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:00.724020 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ea3fd-13fe-4363-b4c4-880f87a1ab6e None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:00.724318 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ea3fd-13fe-4363-b4c4-880f87a1ab6e None admin] PATH_INFO: `/v3/services` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:00.724936 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8ea3fd-13fe-4363-b4c4-880f87a1ab6e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:00.725235 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8ea3fd-13fe-4363-b4c4-880f87a1ab6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:00.731563 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 421/819] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 01:01:00 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 01:01:00.731710 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8ea3fd-13fe-4363-b4c4-880f87a1ab6e None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:00.731789 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1e8b2a-788f-41a4-a9d5-ce5c80d4b030 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:00.731789 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1e8b2a-788f-41a4-a9d5-ce5c80d4b030 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:00.731789 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1e8b2a-788f-41a4-a9d5-ce5c80d4b030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:00.739962 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 399/820] 10.222.0.42 () {56 vars in 1124 bytes} [Thu Sep 8 01:01:00 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:00.739962 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ea3fd-13fe-4363-b4c4-880f87a1ab6e None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:00.755437 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-8a1e8b2a-788f-41a4-a9d5-ce5c80d4b030 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:01:00.756650 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-8a1e8b2a-788f-41a4-a9d5-ce5c80d4b030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:01:00.783584 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:00.828283 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 422/821] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:01:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 01:01:00.829371 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1e8b2a-788f-41a4-a9d5-ce5c80d4b030 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:00.949418 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1e8b2a-788f-41a4-a9d5-ce5c80d4b030 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:00.980877 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:00.983372 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ea3fd-13fe-4363-b4c4-880f87a1ab6e None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:00.995204 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294114c5-8434-4400-b01b-e6ea9917df6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:00.996857 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294114c5-8434-4400-b01b-e6ea9917df6e None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:00.997293 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294114c5-8434-4400-b01b-e6ea9917df6e None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:00.997673 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294114c5-8434-4400-b01b-e6ea9917df6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:01.008750 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294114c5-8434-4400-b01b-e6ea9917df6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:01.010367 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294114c5-8434-4400-b01b-e6ea9917df6e None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:01.025594 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92ca320-af85-4836-b02e-5cc9d9a73d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:01.026247 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ca320-af85-4836-b02e-5cc9d9a73d5b None admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:01.027088 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ca320-af85-4836-b02e-5cc9d9a73d5b None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:01.027521 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ca320-af85-4836-b02e-5cc9d9a73d5b None admin] PATH_INFO: `/v3/projects/78ad2a8fb5264fb4af935e039d28c8de/users/de75fd7d7721433684cafa2e6d3faaae/roles/609d73f56e6a48698397131168abe6df` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:01.027779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-294114c5-8434-4400-b01b-e6ea9917df6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:01:01.027779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:01.027779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:01.027779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:01.027779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:01.027779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:01.027779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:01.027779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:01.027779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:01.027779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:01.027779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:01.027779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:01.027779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:01:01.027779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:01:01.027779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:01:01.027779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:01:01.028868 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:01.028868 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:01.028868 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:01.028868 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:01.028868 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:01.028868 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:01.028868 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:01.028868 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:01.028868 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:01.028868 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:01.028868 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:01.028868 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:01.028868 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:01.028868 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:01.028868 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:01.028868 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:01.028868 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:01:01.029960 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:01:01.029960 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:01:01.029960 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:01.035520 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 423/822] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:01:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:01:01.036434 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294114c5-8434-4400-b01b-e6ea9917df6e None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:01.045706 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92ca320-af85-4836-b02e-5cc9d9a73d5b None admin] RBAC: Authorizing `identity:create_grant(project_id=78ad2a8fb5264fb4af935e039d28c8de, user_id=de75fd7d7721433684cafa2e6d3faaae, role_id=609d73f56e6a48698397131168abe6df)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:01.047765 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92ca320-af85-4836-b02e-5cc9d9a73d5b None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:01.066072 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 400/823] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 8 01:01:00 2022] PUT /identity/v3/projects/78ad2a8fb5264fb4af935e039d28c8de/users/de75fd7d7721433684cafa2e6d3faaae/roles/609d73f56e6a48698397131168abe6df => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:01:01.067026 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92ca320-af85-4836-b02e-5cc9d9a73d5b None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:01.181505 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294114c5-8434-4400-b01b-e6ea9917df6e None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:01.227545 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99354a28-a5c4-4655-b55e-2d97fb434045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:01.228312 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99354a28-a5c4-4655-b55e-2d97fb434045 None admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:01.228714 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99354a28-a5c4-4655-b55e-2d97fb434045 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:01.228994 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99354a28-a5c4-4655-b55e-2d97fb434045 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:01.230761 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99354a28-a5c4-4655-b55e-2d97fb434045 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:01.233267 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99354a28-a5c4-4655-b55e-2d97fb434045 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:01.239768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:01.260205 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92ca320-af85-4836-b02e-5cc9d9a73d5b None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:01.262607 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 424/824] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:01:01 2022] POST /identity/v3/endpoints => generated 346 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:01:01.263531 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99354a28-a5c4-4655-b55e-2d97fb434045 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:01.303406 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:01.313240 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5bca84-2914-4bda-8f0a-02baa52479bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:01.313859 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5bca84-2914-4bda-8f0a-02baa52479bf None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:01.314132 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5bca84-2914-4bda-8f0a-02baa52479bf None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:01.314471 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5bca84-2914-4bda-8f0a-02baa52479bf None admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:01.315132 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5bca84-2914-4bda-8f0a-02baa52479bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:01.315473 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5bca84-2914-4bda-8f0a-02baa52479bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:01.317886 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5bca84-2914-4bda-8f0a-02baa52479bf None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:01.332859 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 401/825] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:01:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:01.340033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5bca84-2914-4bda-8f0a-02baa52479bf None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:01.434278 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99354a28-a5c4-4655-b55e-2d97fb434045 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:01.456913 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5bca84-2914-4bda-8f0a-02baa52479bf None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:01.496081 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72be37c6-97b4-43fe-a3ea-c334ab72ca82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:01.496870 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be37c6-97b4-43fe-a3ea-c334ab72ca82 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:01.497162 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be37c6-97b4-43fe-a3ea-c334ab72ca82 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:01.497430 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be37c6-97b4-43fe-a3ea-c334ab72ca82 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:01.506791 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72be37c6-97b4-43fe-a3ea-c334ab72ca82 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:01.510663 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae453324-3f1d-433d-b955-d1fd40319ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:01.511855 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae453324-3f1d-433d-b955-d1fd40319ab1 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:01.512147 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae453324-3f1d-433d-b955-d1fd40319ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:01.512427 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae453324-3f1d-433d-b955-d1fd40319ab1 None admin] PATH_INFO: `/v3/services/network` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:01.513025 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72be37c6-97b4-43fe-a3ea-c334ab72ca82 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:01.513130 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae453324-3f1d-433d-b955-d1fd40319ab1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:01.514369 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae453324-3f1d-433d-b955-d1fd40319ab1 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:01.524153 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-ae453324-3f1d-433d-b955-d1fd40319ab1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 01:01:01.524153 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:01.524153 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 01:01:01.524153 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 01:01:01.524153 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 01:01:01.524153 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 01:01:01.524153 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 01:01:01.524153 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 01:01:01.524153 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 01:01:01.524153 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:01.524153 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 01:01:01.524153 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:01.524153 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:01.524153 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:01.524153 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:01.524153 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:01.524153 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:01.525172 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:01.525172 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:01.525172 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:01.525172 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:01.525172 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:01.525172 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:01.525172 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 01:01:01.525172 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 01:01:01.525172 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 01:01:01.525172 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 01:01:01.525172 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:01.525172 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:01.525172 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:01.525172 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:01.525172 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:01.525172 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:01.526016 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:01.526016 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:01.526016 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:01.526016 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:01.526016 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:01.526016 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:01.526016 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:01.526016 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:01.526016 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:01.526016 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:01.526016 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 01:01:01.526016 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 01:01:01.526016 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 01:01:01.526016 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:01.529728 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 425/826] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 01:01:01 2022] GET /identity/v3/services/network => generated 88 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:01:01.535493 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae453324-3f1d-433d-b955-d1fd40319ab1 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:01.542768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-72be37c6-97b4-43fe-a3ea-c334ab72ca82 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:01:01.542768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:01.542768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:01.542768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:01.542768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:01.542768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:01.542768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:01.542768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:01.542768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:01.542768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:01.542768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:01.542768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:01.542768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 01:01:01.542768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 01:01:01.542768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 01:01:01.544037 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 01:01:01.544037 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:01.544037 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:01.544037 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:01.544037 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:01.544037 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:01.544037 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:01.544037 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:01.544037 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:01.544037 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:01.544037 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:01.544037 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:01.544037 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:01.544037 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:01.544037 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:01.544037 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:01.544037 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:01.545465 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 01:01:01.545465 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 01:01:01.545465 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 01:01:01.545465 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 01:01:01.545465 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 01:01:01.545465 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 01:01:01.545465 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:01:01.545465 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:01.545465 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 402/827] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 01:01:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:01:01.552054 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72be37c6-97b4-43fe-a3ea-c334ab72ca82 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:01.564312 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:01.656450 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:01.664128 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae453324-3f1d-433d-b955-d1fd40319ab1 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:01.678518 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72be37c6-97b4-43fe-a3ea-c334ab72ca82 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:01.700418 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cca1c1-b52e-471b-a42a-1f2bf9934bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:01.701268 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cca1c1-b52e-471b-a42a-1f2bf9934bb6 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:01.701685 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cca1c1-b52e-471b-a42a-1f2bf9934bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:01.702133 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cca1c1-b52e-471b-a42a-1f2bf9934bb6 None admin] PATH_INFO: `/v3/services` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:01.702911 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cca1c1-b52e-471b-a42a-1f2bf9934bb6 None admin] RBAC: Adding query filter params (name=network) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:01.703370 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cca1c1-b52e-471b-a42a-1f2bf9934bb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:01.704945 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cca1c1-b52e-471b-a42a-1f2bf9934bb6 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:01.710228 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 426/828] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Sep 8 01:01:01 2022] GET /identity/v3/services?name=network => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:01:01.716531 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cca1c1-b52e-471b-a42a-1f2bf9934bb6 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:01.740730 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3a892e-c038-4efd-b62e-c72448b81094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:01.741340 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a892e-c038-4efd-b62e-c72448b81094 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:01.741613 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a892e-c038-4efd-b62e-c72448b81094 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:01.741881 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a892e-c038-4efd-b62e-c72448b81094 None admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:01.742634 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3a892e-c038-4efd-b62e-c72448b81094 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:01.742959 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3a892e-c038-4efd-b62e-c72448b81094 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:01.746894 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3a892e-c038-4efd-b62e-c72448b81094 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:01.760742 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 403/829] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 8 01:01:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:01.779718 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3a892e-c038-4efd-b62e-c72448b81094 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:01.834689 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cca1c1-b52e-471b-a42a-1f2bf9934bb6 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:01.871952 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59374773-ea4f-4f96-ad0b-31b1b7aa021d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:01.872890 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59374773-ea4f-4f96-ad0b-31b1b7aa021d None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:01.873355 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59374773-ea4f-4f96-ad0b-31b1b7aa021d None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:01.873808 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59374773-ea4f-4f96-ad0b-31b1b7aa021d None admin] PATH_INFO: `/v3/services` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:01.874648 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59374773-ea4f-4f96-ad0b-31b1b7aa021d None admin] RBAC: Adding query filter params (type=network) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:01.875134 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59374773-ea4f-4f96-ad0b-31b1b7aa021d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:01.876712 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59374773-ea4f-4f96-ad0b-31b1b7aa021d None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:01.882588 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 427/830] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Sep 8 01:01:01 2022] GET /identity/v3/services?type=network => generated 350 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:01.889047 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59374773-ea4f-4f96-ad0b-31b1b7aa021d None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:01.916488 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:01.945555 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:01.992986 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3a892e-c038-4efd-b62e-c72448b81094 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:02.013691 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59374773-ea4f-4f96-ad0b-31b1b7aa021d None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:02.037131 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d85cf-e6a4-4c9e-9d1d-ad5e7880a70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:02.038114 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d85cf-e6a4-4c9e-9d1d-ad5e7880a70b None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:02.038617 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d85cf-e6a4-4c9e-9d1d-ad5e7880a70b None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:02.039088 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d85cf-e6a4-4c9e-9d1d-ad5e7880a70b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:02.040009 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8d85cf-e6a4-4c9e-9d1d-ad5e7880a70b None admin] RBAC: Adding query filter params (role.id=34de369ec7c54715aa83a9052a26c3f2, scope.project.id=78ad2a8fb5264fb4af935e039d28c8de, user.id=ae643cfdcb164aa9b474747cd21aa116) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:02.040542 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8d85cf-e6a4-4c9e-9d1d-ad5e7880a70b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:02.042832 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8d85cf-e6a4-4c9e-9d1d-ad5e7880a70b None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:02.049844 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 404/831] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 8 01:01:01 2022] GET /identity/v3/role_assignments?user.id=ae643cfdcb164aa9b474747cd21aa116&scope.project.id=78ad2a8fb5264fb4af935e039d28c8de&role.id=34de369ec7c54715aa83a9052a26c3f2 => generated 608 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:01:02.051411 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6fed1c-0a39-4f9c-ab47-7788a67eb98e None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:02.051894 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6fed1c-0a39-4f9c-ab47-7788a67eb98e None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:02.052405 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6fed1c-0a39-4f9c-ab47-7788a67eb98e None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:02.053946 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 405/832] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:01:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:01:02.056294 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12b44b3-e973-4b6f-a60a-89cf285f6872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:02.057299 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b44b3-e973-4b6f-a60a-89cf285f6872 None admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:02.058378 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b44b3-e973-4b6f-a60a-89cf285f6872 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:02.058928 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b44b3-e973-4b6f-a60a-89cf285f6872 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:02.059881 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12b44b3-e973-4b6f-a60a-89cf285f6872 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:02.061426 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12b44b3-e973-4b6f-a60a-89cf285f6872 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:02.061863 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f46df7-f79f-421a-bd41-5b49e673fa4e None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:02.062368 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f46df7-f79f-421a-bd41-5b49e673fa4e None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:02.062835 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f46df7-f79f-421a-bd41-5b49e673fa4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:02.090132 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-88f46df7-f79f-421a-bd41-5b49e673fa4e None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:01:02.091420 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-88f46df7-f79f-421a-bd41-5b49e673fa4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:01:02.101612 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 428/833] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:01:01 2022] POST /identity/v3/endpoints => generated 337 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:01:02.223683 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 406/834] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:01:02 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 01:01:02.534630 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12b44b3-e973-4b6f-a60a-89cf285f6872 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:02.614177 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:02.652322 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12b44b3-e973-4b6f-a60a-89cf285f6872 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:02.687871 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27912a89-f5f6-416a-9504-39fffbd19bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:02.688528 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27912a89-f5f6-416a-9504-39fffbd19bb3 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:02.688803 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27912a89-f5f6-416a-9504-39fffbd19bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:02.689068 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27912a89-f5f6-416a-9504-39fffbd19bb3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:02.699639 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27912a89-f5f6-416a-9504-39fffbd19bb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:01:02.705539 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-27912a89-f5f6-416a-9504-39fffbd19bb3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 01:01:02.705539 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:02.705539 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:02.705539 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:02.705539 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:02.705539 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:02.705539 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:02.705539 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:02.705539 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:02.705539 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:02.705539 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:02.705539 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:02.705539 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:01:02.705539 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:01:02.705539 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:01:02.705539 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:01:02.706592 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:01:02.706592 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:01:02.706592 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:01:02.706592 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:01:02.706592 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:02.706592 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:02.706592 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:02.706592 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:02.706592 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:02.706592 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:02.706592 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:02.706592 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:02.706592 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:02.706592 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:02.706592 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:02.706592 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:02.707434 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:02.707434 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:02.707434 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:02.707434 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:02.707434 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:01:02.707434 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:01:02.707434 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:01:02.707434 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:01:02.707434 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:01:02.707434 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:01:02.707434 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 01:01:02.707434 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:02.707434 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 429/835] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 8 01:01:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:01:02.712740 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f46df7-f79f-421a-bd41-5b49e673fa4e None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:02.770208 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:02.816151 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f46df7-f79f-421a-bd41-5b49e673fa4e None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:02.851594 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a196d17-2da7-49c7-99b7-a53c94046b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:02.852272 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a196d17-2da7-49c7-99b7-a53c94046b93 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:02.852550 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a196d17-2da7-49c7-99b7-a53c94046b93 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:02.852820 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a196d17-2da7-49c7-99b7-a53c94046b93 None admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:02.853483 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a196d17-2da7-49c7-99b7-a53c94046b93 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:02.853782 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a196d17-2da7-49c7-99b7-a53c94046b93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:02.855191 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a196d17-2da7-49c7-99b7-a53c94046b93 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:02.863675 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 407/836] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 01:01:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:02.869252 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27912a89-f5f6-416a-9504-39fffbd19bb3 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:02.975533 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27912a89-f5f6-416a-9504-39fffbd19bb3 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:03.013457 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c408b1e-a741-4611-8144-4162d35ca44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:03.014081 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c408b1e-a741-4611-8144-4162d35ca44f None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:03.014353 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c408b1e-a741-4611-8144-4162d35ca44f None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:03.014618 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c408b1e-a741-4611-8144-4162d35ca44f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:03.022021 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:03.026243 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c408b1e-a741-4611-8144-4162d35ca44f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:03.028279 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c408b1e-a741-4611-8144-4162d35ca44f None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:03.043635 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-0c408b1e-a741-4611-8144-4162d35ca44f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:01:03.043635 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:03.043635 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:03.043635 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:03.043635 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:03.043635 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:03.043635 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:03.043635 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:03.043635 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:03.043635 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:03.043635 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:03.043635 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:03.043635 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:01:03.043635 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:01:03.043635 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:01:03.043635 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:01:03.045850 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:03.045850 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:03.045850 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:03.045850 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:03.045850 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:03.045850 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:03.045850 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:03.045850 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:03.045850 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:03.045850 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:03.045850 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:03.045850 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:03.045850 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:03.045850 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:03.045850 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:03.045850 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:03.045850 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:01:03.048035 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:01:03.048035 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:01:03.048035 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:03.048035 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 430/837] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:01:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:01:03.053220 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a196d17-2da7-49c7-99b7-a53c94046b93 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:03.158695 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a196d17-2da7-49c7-99b7-a53c94046b93 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:03.177646 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:03.197320 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a13e7c4-cbfe-4968-8474-82ee57e41371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:03.197930 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13e7c4-cbfe-4968-8474-82ee57e41371 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:03.198205 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13e7c4-cbfe-4968-8474-82ee57e41371 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:03.198659 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13e7c4-cbfe-4968-8474-82ee57e41371 None admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:03.199394 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a13e7c4-cbfe-4968-8474-82ee57e41371 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:03.199694 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a13e7c4-cbfe-4968-8474-82ee57e41371 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:03.201110 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a13e7c4-cbfe-4968-8474-82ee57e41371 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:03.218131 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 408/838] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:01:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:03.223343 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c408b1e-a741-4611-8144-4162d35ca44f None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:03.329448 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c408b1e-a741-4611-8144-4162d35ca44f None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:03.365261 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02909e37-2099-4cab-bde3-f7c4df7702f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:03.365928 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02909e37-2099-4cab-bde3-f7c4df7702f8 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:03.366205 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02909e37-2099-4cab-bde3-f7c4df7702f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:03.366629 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02909e37-2099-4cab-bde3-f7c4df7702f8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:03.374402 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02909e37-2099-4cab-bde3-f7c4df7702f8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:03.375982 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02909e37-2099-4cab-bde3-f7c4df7702f8 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:03.387842 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-02909e37-2099-4cab-bde3-f7c4df7702f8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 08 01:01:03.387842 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:03.387842 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:03.387842 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:03.387842 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:03.387842 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:03.387842 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:03.387842 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:03.387842 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:03.387842 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:03.387842 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:03.387842 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:03.387842 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 01:01:03.387842 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 01:01:03.387842 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 01:01:03.387842 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 01:01:03.390005 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:03.390005 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:03.390005 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 01:01:03.390005 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:01:03.390005 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 01:01:03.390005 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:01:03.390005 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:03.390005 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:03.390005 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:03.390005 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:03.390005 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:03.390005 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:03.390005 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:03.390005 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:03.390005 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:03.390005 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:03.390005 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:03.390005 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:03.392268 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:03.392268 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:03.392268 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 01:01:03.392268 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 01:01:03.392268 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 01:01:03.392268 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 01:01:03.392268 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 01:01:03.392268 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 01:01:03.392268 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 08 01:01:03.392268 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:03.392268 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 431/839] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 8 01:01:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:01:03.397826 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a13e7c4-cbfe-4968-8474-82ee57e41371 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:03.505922 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a13e7c4-cbfe-4968-8474-82ee57e41371 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:03.545999 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d76772-a9e4-403c-8070-a0f80414e638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:03.546648 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d76772-a9e4-403c-8070-a0f80414e638 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:03.546928 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d76772-a9e4-403c-8070-a0f80414e638 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:03.547199 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d76772-a9e4-403c-8070-a0f80414e638 None admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:03.547994 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d76772-a9e4-403c-8070-a0f80414e638 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:03.548327 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d76772-a9e4-403c-8070-a0f80414e638 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:03.549774 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d76772-a9e4-403c-8070-a0f80414e638 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:03.569584 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 409/840] 10.222.0.42 () {56 vars in 1135 bytes} [Thu Sep 8 01:01:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:01:03.575293 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02909e37-2099-4cab-bde3-f7c4df7702f8 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:03.632168 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:03.638892 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ea402-cf9f-4e5a-8331-63866eb43516 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:03.639171 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ea402-cf9f-4e5a-8331-63866eb43516 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:03.639438 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ea402-cf9f-4e5a-8331-63866eb43516 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:03.644167 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 410/841] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:01:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 01:01:03.646259 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d1754f-7533-491e-a751-d7eb7e0f681f None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:03.646822 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d1754f-7533-491e-a751-d7eb7e0f681f None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:03.647297 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d1754f-7533-491e-a751-d7eb7e0f681f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:03.669682 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-73d1754f-7533-491e-a751-d7eb7e0f681f None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:01:03.671963 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-73d1754f-7533-491e-a751-d7eb7e0f681f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:01:03.693583 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02909e37-2099-4cab-bde3-f7c4df7702f8 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:03.710845 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:03.740763 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d830fb94-3179-4726-a140-fcfdb52bf0b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:03.741380 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d830fb94-3179-4726-a140-fcfdb52bf0b9 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:03.741656 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d830fb94-3179-4726-a140-fcfdb52bf0b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:03.741984 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d830fb94-3179-4726-a140-fcfdb52bf0b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:03.783804 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 411/842] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:01:03 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 01:01:03.822692 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d830fb94-3179-4726-a140-fcfdb52bf0b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:03.824729 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d830fb94-3179-4726-a140-fcfdb52bf0b9 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:03.835805 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8f1620-abd3-453f-9feb-d644f8ca6cbb None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:03.836081 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8f1620-abd3-453f-9feb-d644f8ca6cbb None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:03.836401 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8f1620-abd3-453f-9feb-d644f8ca6cbb None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:03.837969 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 412/843] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:01:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 01:01:03.842929 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289364c-0e4a-4366-b0b4-3a53190e1e8a None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:03.843212 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289364c-0e4a-4366-b0b4-3a53190e1e8a None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:03.843484 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289364c-0e4a-4366-b0b4-3a53190e1e8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:03.865039 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-9289364c-0e4a-4366-b0b4-3a53190e1e8a None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:01:03.866246 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-9289364c-0e4a-4366-b0b4-3a53190e1e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:01:03.880480 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:03.936828 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 432/844] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 01:01:03 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 01:01:03.942856 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d830fb94-3179-4726-a140-fcfdb52bf0b9 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:03.954386 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 413/845] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:01:03 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 01:01:04.053163 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d830fb94-3179-4726-a140-fcfdb52bf0b9 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:04.072245 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9289364c-0e4a-4366-b0b4-3a53190e1e8a None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:04.097272 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9413b6-2034-4d28-901b-42bdd8b06152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:04.098261 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9413b6-2034-4d28-901b-42bdd8b06152 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:04.099295 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9413b6-2034-4d28-901b-42bdd8b06152 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:04.099770 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9413b6-2034-4d28-901b-42bdd8b06152 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:04.112385 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9413b6-2034-4d28-901b-42bdd8b06152 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:04.115252 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9413b6-2034-4d28-901b-42bdd8b06152 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:04.133919 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-ac9413b6-2034-4d28-901b-42bdd8b06152 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:01:04.133919 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:04.133919 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:04.133919 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:04.133919 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:04.133919 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:04.133919 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:04.133919 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:04.133919 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:04.133919 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:04.133919 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:04.133919 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:04.133919 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:01:04.133919 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:01:04.133919 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:01:04.133919 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:01:04.134910 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:04.134910 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:04.134910 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:04.134910 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:04.134910 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:04.134910 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:04.134910 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:04.134910 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:04.134910 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:04.134910 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:04.134910 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:04.134910 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:04.134910 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:04.134910 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:04.134910 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:04.134910 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:04.134910 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:01:04.135914 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:01:04.135914 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:01:04.135914 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:04.141970 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 433/846] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:01:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:01:04.143004 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9413b6-2034-4d28-901b-42bdd8b06152 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:04.146878 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:04.197892 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:04.206196 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9289364c-0e4a-4366-b0b4-3a53190e1e8a None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:04.246128 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6abd4d9-85b7-48ad-a2f6-18e50f34f821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:04.246775 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6abd4d9-85b7-48ad-a2f6-18e50f34f821 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:04.247060 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6abd4d9-85b7-48ad-a2f6-18e50f34f821 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:04.247328 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6abd4d9-85b7-48ad-a2f6-18e50f34f821 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:04.257744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6abd4d9-85b7-48ad-a2f6-18e50f34f821 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:01:04.263316 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-c6abd4d9-85b7-48ad-a2f6-18e50f34f821 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 01:01:04.263316 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:04.263316 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:04.263316 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:04.263316 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:04.263316 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:04.263316 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:04.263316 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:04.263316 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:04.263316 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:04.263316 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:04.263316 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:04.263316 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:01:04.263316 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:01:04.263316 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:01:04.263316 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:01:04.264284 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:01:04.264284 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:01:04.264284 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:01:04.264284 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:01:04.264284 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:04.264284 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:04.264284 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:04.264284 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:04.264284 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:04.264284 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:04.264284 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:04.264284 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:04.264284 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:04.264284 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:04.264284 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:04.264284 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:04.265147 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:04.265147 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:04.265147 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:04.265147 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:04.265147 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:01:04.265147 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:01:04.265147 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:01:04.265147 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:01:04.265147 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:01:04.265147 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:01:04.265147 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 01:01:04.265147 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:04.265792 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9413b6-2034-4d28-901b-42bdd8b06152 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:04.268276 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 414/847] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 8 01:01:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:01:04.269368 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6abd4d9-85b7-48ad-a2f6-18e50f34f821 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:04.300577 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127e76b5-f598-4524-981b-2ae1ba9e085b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:04.301262 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e76b5-f598-4524-981b-2ae1ba9e085b None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:04.301551 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e76b5-f598-4524-981b-2ae1ba9e085b None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:04.301818 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e76b5-f598-4524-981b-2ae1ba9e085b None admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:04.302456 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127e76b5-f598-4524-981b-2ae1ba9e085b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:04.302754 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127e76b5-f598-4524-981b-2ae1ba9e085b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:04.304175 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127e76b5-f598-4524-981b-2ae1ba9e085b None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:04.316876 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 434/848] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:01:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:04.318098 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127e76b5-f598-4524-981b-2ae1ba9e085b None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:04.380598 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6abd4d9-85b7-48ad-a2f6-18e50f34f821 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:04.389965 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:04.422289 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a090d2a-c322-42da-96ea-ce637221c933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:04.422923 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a090d2a-c322-42da-96ea-ce637221c933 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:04.423200 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a090d2a-c322-42da-96ea-ce637221c933 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:04.423470 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a090d2a-c322-42da-96ea-ce637221c933 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:04.424053 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a090d2a-c322-42da-96ea-ce637221c933 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:04.425499 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a090d2a-c322-42da-96ea-ce637221c933 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:04.434504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-3a090d2a-c322-42da-96ea-ce637221c933 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 01:01:04.434504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:04.434504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 01:01:04.434504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 01:01:04.434504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 01:01:04.434504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 01:01:04.434504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 01:01:04.434504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 01:01:04.434504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 01:01:04.434504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:04.434504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 01:01:04.434504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:04.434504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:04.434504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:04.434504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:04.434504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:04.434504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:04.435513 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:04.435513 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:04.435513 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:04.435513 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:04.435513 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:04.435513 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:04.435513 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 01:01:04.435513 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 01:01:04.435513 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 01:01:04.435513 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 01:01:04.435513 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:04.435513 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:04.435513 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:04.435513 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:04.435513 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:04.435513 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:04.436413 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:04.436413 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:04.436413 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:04.436413 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:04.436413 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:04.436413 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:04.436413 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:04.436413 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:04.436413 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:04.436413 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:04.436413 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 01:01:04.436413 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 01:01:04.436413 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 01:01:04.436413 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:04.436413 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 415/849] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 8 01:01:04 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:01:04.437229 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a090d2a-c322-42da-96ea-ce637221c933 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:04.442276 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127e76b5-f598-4524-981b-2ae1ba9e085b None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:04.460895 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:04.482745 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedde1e0-1ced-41d8-991e-70b10942cd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:04.483356 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedde1e0-1ced-41d8-991e-70b10942cd2c None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:04.483636 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedde1e0-1ced-41d8-991e-70b10942cd2c None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:04.484363 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedde1e0-1ced-41d8-991e-70b10942cd2c None admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:04.485042 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedde1e0-1ced-41d8-991e-70b10942cd2c None admin] RBAC: Adding query filter params (name=service) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:04.485351 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedde1e0-1ced-41d8-991e-70b10942cd2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:04.487110 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedde1e0-1ced-41d8-991e-70b10942cd2c None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:04.497178 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 435/850] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 8 01:01:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:04.498514 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedde1e0-1ced-41d8-991e-70b10942cd2c None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:04.547675 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a090d2a-c322-42da-96ea-ce637221c933 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:04.581073 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c302b403-f1cc-4fad-a686-dd3a684bf182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:04.581629 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c302b403-f1cc-4fad-a686-dd3a684bf182 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:04.581903 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c302b403-f1cc-4fad-a686-dd3a684bf182 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:04.582175 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c302b403-f1cc-4fad-a686-dd3a684bf182 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:04.588651 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c302b403-f1cc-4fad-a686-dd3a684bf182 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:04.590812 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c302b403-f1cc-4fad-a686-dd3a684bf182 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:04.599383 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedde1e0-1ced-41d8-991e-70b10942cd2c None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:04.604681 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-c302b403-f1cc-4fad-a686-dd3a684bf182 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:01:04.604681 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:04.604681 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:04.604681 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:04.604681 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:04.604681 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:04.604681 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:04.604681 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:04.604681 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:04.604681 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:04.604681 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:04.604681 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:04.604681 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 01:01:04.604681 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 01:01:04.604681 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 01:01:04.605726 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 01:01:04.605726 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:04.605726 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:04.605726 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:04.605726 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:04.605726 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:04.605726 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:04.605726 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:04.605726 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:04.605726 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:04.605726 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:04.605726 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:04.605726 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:04.605726 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:04.605726 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:04.605726 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:04.605726 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:04.606613 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 01:01:04.606613 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 01:01:04.606613 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 01:01:04.606613 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 01:01:04.606613 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 01:01:04.606613 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 01:01:04.606613 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:01:04.606613 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:04.606613 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 416/851] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 01:01:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:01:04.607163 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79bc0f5-4ba8-4c1c-acc6-6178d12dce03 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:04.607461 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79bc0f5-4ba8-4c1c-acc6-6178d12dce03 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:04.607787 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79bc0f5-4ba8-4c1c-acc6-6178d12dce03 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:04.609095 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 417/852] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:01:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:01:04.610989 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79bc0f5-4ba8-4c1c-acc6-6178d12dce03 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:04.625193 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:04.639848 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7650686-6696-4625-98b9-03c5b60f58d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:04.640492 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7650686-6696-4625-98b9-03c5b60f58d2 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:04.640775 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7650686-6696-4625-98b9-03c5b60f58d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:04.641047 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7650686-6696-4625-98b9-03c5b60f58d2 None admin] PATH_INFO: `/v3/services` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:04.641712 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7650686-6696-4625-98b9-03c5b60f58d2 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:04.642013 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7650686-6696-4625-98b9-03c5b60f58d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:04.643405 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7650686-6696-4625-98b9-03c5b60f58d2 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:04.648159 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 436/853] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 8 01:01:04 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:04.649081 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7650686-6696-4625-98b9-03c5b60f58d2 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:04.694842 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:04.718114 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79bc0f5-4ba8-4c1c-acc6-6178d12dce03 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:04.750952 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226296f8-29cd-4b18-81f5-5f0d61564021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:04.751426 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7650686-6696-4625-98b9-03c5b60f58d2 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:04.751570 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226296f8-29cd-4b18-81f5-5f0d61564021 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:04.751845 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226296f8-29cd-4b18-81f5-5f0d61564021 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:04.752112 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226296f8-29cd-4b18-81f5-5f0d61564021 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:04.762995 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226296f8-29cd-4b18-81f5-5f0d61564021 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:04.764458 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226296f8-29cd-4b18-81f5-5f0d61564021 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:04.779106 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-226296f8-29cd-4b18-81f5-5f0d61564021 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:01:04.779106 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:04.779106 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:04.779106 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:04.779106 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:04.779106 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:04.779106 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:04.779106 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:04.779106 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:04.779106 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:04.779106 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:04.779106 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:04.779106 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:01:04.779106 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:01:04.779106 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:01:04.779106 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:01:04.780060 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:04.780060 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:04.780060 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:04.780060 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:04.780060 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:04.780060 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:04.780060 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:04.780060 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:04.780060 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:04.780060 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:04.780060 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:04.780060 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:04.780060 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:04.780060 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:04.780060 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:04.780060 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:04.780060 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:01:04.781149 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:01:04.781149 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:01:04.781149 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:04.781149 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 418/854] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:01:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:01:04.781622 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5383ba-4c47-46b3-97f1-018b0f7b691b None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:04.781622 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5383ba-4c47-46b3-97f1-018b0f7b691b None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:04.781842 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5383ba-4c47-46b3-97f1-018b0f7b691b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:04.788185 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40510e37-1653-45f0-b56b-980f57bebb3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:04.788962 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40510e37-1653-45f0-b56b-980f57bebb3b None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:04.790026 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40510e37-1653-45f0-b56b-980f57bebb3b None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:04.790307 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40510e37-1653-45f0-b56b-980f57bebb3b None admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:04.791004 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40510e37-1653-45f0-b56b-980f57bebb3b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:04.791311 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40510e37-1653-45f0-b56b-980f57bebb3b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:04.792820 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40510e37-1653-45f0-b56b-980f57bebb3b None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:04.799112 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-bc5383ba-4c47-46b3-97f1-018b0f7b691b None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:01:04.800341 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-bc5383ba-4c47-46b3-97f1-018b0f7b691b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:01:04.805641 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 437/855] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 8 01:01:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:04.806508 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40510e37-1653-45f0-b56b-980f57bebb3b None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:04.855002 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:04.871169 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 419/856] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:01:04 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 01:01:04.872300 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5383ba-4c47-46b3-97f1-018b0f7b691b None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:04.916650 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40510e37-1653-45f0-b56b-980f57bebb3b None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:04.937140 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:04.954718 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa55f2-4e6f-4c67-8194-3126cf9ecd42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:04.955150 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa55f2-4e6f-4c67-8194-3126cf9ecd42 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:04.955456 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa55f2-4e6f-4c67-8194-3126cf9ecd42 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:04.955728 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa55f2-4e6f-4c67-8194-3126cf9ecd42 None admin] PATH_INFO: `/v3/services` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:04.956384 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aa55f2-4e6f-4c67-8194-3126cf9ecd42 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:04.956691 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aa55f2-4e6f-4c67-8194-3126cf9ecd42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:04.958056 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aa55f2-4e6f-4c67-8194-3126cf9ecd42 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:04.972164 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 438/857] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 8 01:01:04 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:04.972164 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa55f2-4e6f-4c67-8194-3126cf9ecd42 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:04.993095 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5383ba-4c47-46b3-97f1-018b0f7b691b None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:05.032592 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d26ed52-1f34-4082-a679-e181a15cf653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:05.033364 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d26ed52-1f34-4082-a679-e181a15cf653 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:05.033703 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d26ed52-1f34-4082-a679-e181a15cf653 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:05.034051 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d26ed52-1f34-4082-a679-e181a15cf653 None admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:05.034778 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d26ed52-1f34-4082-a679-e181a15cf653 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:05.035197 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d26ed52-1f34-4082-a679-e181a15cf653 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:05.036766 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d26ed52-1f34-4082-a679-e181a15cf653 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:05.051055 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 420/858] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:01:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:05.056269 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d26ed52-1f34-4082-a679-e181a15cf653 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:05.106219 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa55f2-4e6f-4c67-8194-3126cf9ecd42 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:05.126066 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:05.162208 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308bc6c2-050f-4f1a-af3a-c4d3ffbb2b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:05.162815 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308bc6c2-050f-4f1a-af3a-c4d3ffbb2b15 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:05.163088 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308bc6c2-050f-4f1a-af3a-c4d3ffbb2b15 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:05.163351 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308bc6c2-050f-4f1a-af3a-c4d3ffbb2b15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:05.164242 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308bc6c2-050f-4f1a-af3a-c4d3ffbb2b15 None admin] RBAC: Adding query filter params (role.id=34de369ec7c54715aa83a9052a26c3f2, scope.project.id=78ad2a8fb5264fb4af935e039d28c8de, user.id=8add26872fe6479aa9d7de155a17195a) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:05.164551 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308bc6c2-050f-4f1a-af3a-c4d3ffbb2b15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:05.165992 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308bc6c2-050f-4f1a-af3a-c4d3ffbb2b15 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:05.172639 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 439/859] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 8 01:01:04 2022] GET /identity/v3/role_assignments?user.id=8add26872fe6479aa9d7de155a17195a&scope.project.id=78ad2a8fb5264fb4af935e039d28c8de&role.id=34de369ec7c54715aa83a9052a26c3f2 => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:05.173435 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308bc6c2-050f-4f1a-af3a-c4d3ffbb2b15 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:05.177991 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d26ed52-1f34-4082-a679-e181a15cf653 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:05.208429 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:05.223720 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca98554-36b8-444b-a2dd-208f2a0aaba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:05.225280 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca98554-36b8-444b-a2dd-208f2a0aaba3 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:05.225772 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca98554-36b8-444b-a2dd-208f2a0aaba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:05.226659 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca98554-36b8-444b-a2dd-208f2a0aaba3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:05.238961 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca98554-36b8-444b-a2dd-208f2a0aaba3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:05.241958 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca98554-36b8-444b-a2dd-208f2a0aaba3 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:05.265791 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-eca98554-36b8-444b-a2dd-208f2a0aaba3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 01:01:05.265791 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:05.265791 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:05.265791 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:05.265791 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:05.265791 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:05.265791 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:05.265791 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:05.265791 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:05.265791 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:05.265791 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:05.265791 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:05.265791 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 01:01:05.265791 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 01:01:05.265791 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 01:01:05.265791 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 01:01:05.266818 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:05.266818 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:05.266818 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 01:01:05.266818 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:01:05.266818 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 01:01:05.266818 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:01:05.266818 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:05.266818 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:05.266818 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:05.266818 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:05.266818 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:05.266818 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:05.266818 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:05.266818 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:05.266818 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:05.266818 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:05.266818 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:05.266818 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:05.267759 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:05.267759 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:05.267759 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 01:01:05.267759 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 01:01:05.267759 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 01:01:05.267759 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 01:01:05.267759 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 01:01:05.267759 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 01:01:05.267759 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 01:01:05.267759 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:05.267759 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 421/860] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Sep 8 01:01:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:01:05.276814 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca98554-36b8-444b-a2dd-208f2a0aaba3 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:05.306797 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308bc6c2-050f-4f1a-af3a-c4d3ffbb2b15 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:05.349644 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3356cf8d-936e-47e7-ba2a-0d5577dc006d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:05.350551 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356cf8d-936e-47e7-ba2a-0d5577dc006d None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:05.350980 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356cf8d-936e-47e7-ba2a-0d5577dc006d None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:05.351411 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356cf8d-936e-47e7-ba2a-0d5577dc006d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:05.352284 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3356cf8d-936e-47e7-ba2a-0d5577dc006d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:05.355034 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3356cf8d-936e-47e7-ba2a-0d5577dc006d None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:05.364779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-3356cf8d-936e-47e7-ba2a-0d5577dc006d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 01:01:05.364779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:05.364779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 01:01:05.364779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 01:01:05.364779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 01:01:05.364779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 01:01:05.364779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 01:01:05.364779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 01:01:05.364779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 01:01:05.364779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:05.364779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 01:01:05.364779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:05.364779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:05.364779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:05.364779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:05.364779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:05.365760 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:05.365760 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:05.365760 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:05.365760 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:05.365760 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:05.365760 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:05.365760 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:05.365760 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 01:01:05.365760 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 01:01:05.365760 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 01:01:05.365760 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 01:01:05.365760 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:05.365760 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:05.365760 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:05.365760 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:05.365760 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:05.365760 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:05.366748 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:05.366748 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:05.366748 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:05.366748 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:05.366748 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:05.366748 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:05.366748 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:05.366748 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:05.366748 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:05.366748 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:05.366748 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 01:01:05.366748 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 01:01:05.366748 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 01:01:05.366748 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:05.369682 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 440/861] 10.222.0.42 () {56 vars in 1132 bytes} [Thu Sep 8 01:01:05 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:01:05.377253 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3356cf8d-936e-47e7-ba2a-0d5577dc006d None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:05.407349 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca98554-36b8-444b-a2dd-208f2a0aaba3 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:05.429327 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:05.448710 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b818718-d501-405f-97b8-891d3b64dece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:05.449692 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b818718-d501-405f-97b8-891d3b64dece None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:05.450180 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b818718-d501-405f-97b8-891d3b64dece None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:05.450658 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b818718-d501-405f-97b8-891d3b64dece None admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:05.451613 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b818718-d501-405f-97b8-891d3b64dece None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:05.452154 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b818718-d501-405f-97b8-891d3b64dece None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:05.453820 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b818718-d501-405f-97b8-891d3b64dece None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:05.474687 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 422/862] 10.222.0.42 () {56 vars in 1149 bytes} [Thu Sep 8 01:01:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:01:05.481238 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b818718-d501-405f-97b8-891d3b64dece None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:05.499890 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3356cf8d-936e-47e7-ba2a-0d5577dc006d None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:05.533978 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:05.546937 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e981ca0-198f-4dc1-9c7f-61b9f7180702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:05.547892 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e981ca0-198f-4dc1-9c7f-61b9f7180702 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:05.548383 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e981ca0-198f-4dc1-9c7f-61b9f7180702 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:05.548827 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e981ca0-198f-4dc1-9c7f-61b9f7180702 None admin] PATH_INFO: `/v3/services` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:05.549901 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e981ca0-198f-4dc1-9c7f-61b9f7180702 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:05.550395 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e981ca0-198f-4dc1-9c7f-61b9f7180702 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:05.553282 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e981ca0-198f-4dc1-9c7f-61b9f7180702 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:05.562555 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 441/863] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Sep 8 01:01:05 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:01:05.569592 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e981ca0-198f-4dc1-9c7f-61b9f7180702 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:05.653689 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b818718-d501-405f-97b8-891d3b64dece None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:05.689138 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e981ca0-198f-4dc1-9c7f-61b9f7180702 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:05.691531 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:05.694123 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a671483-e0de-49ba-8b37-b1011603433a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:05.694765 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a671483-e0de-49ba-8b37-b1011603433a None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:05.695038 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a671483-e0de-49ba-8b37-b1011603433a None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:05.695301 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a671483-e0de-49ba-8b37-b1011603433a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:05.725482 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d782ae-aefc-4f62-ae56-d7275b664df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:05.726088 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d782ae-aefc-4f62-ae56-d7275b664df5 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:05.726363 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d782ae-aefc-4f62-ae56-d7275b664df5 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:05.726632 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d782ae-aefc-4f62-ae56-d7275b664df5 None admin] PATH_INFO: `/v3/services` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:05.727324 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d782ae-aefc-4f62-ae56-d7275b664df5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:05.727607 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d782ae-aefc-4f62-ae56-d7275b664df5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:05.729195 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d782ae-aefc-4f62-ae56-d7275b664df5 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:05.740165 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 442/864] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Sep 8 01:01:05 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:05.794612 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a671483-e0de-49ba-8b37-b1011603433a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:05.796212 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a671483-e0de-49ba-8b37-b1011603433a None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:05.844062 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:05.919392 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 423/865] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 01:01:05 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 01:01:05.924911 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d782ae-aefc-4f62-ae56-d7275b664df5 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:06.039612 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d782ae-aefc-4f62-ae56-d7275b664df5 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:06.075782 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc999b-b842-49ab-8ba4-6e9e70be98cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:06.076445 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc999b-b842-49ab-8ba4-6e9e70be98cd None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:06.076725 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc999b-b842-49ab-8ba4-6e9e70be98cd None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:06.076994 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc999b-b842-49ab-8ba4-6e9e70be98cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:06.088058 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fc999b-b842-49ab-8ba4-6e9e70be98cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:06.089596 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fc999b-b842-49ab-8ba4-6e9e70be98cd None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:06.104065 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-d4fc999b-b842-49ab-8ba4-6e9e70be98cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:01:06.104065 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:06.104065 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:06.104065 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:06.104065 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:06.104065 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:06.104065 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:06.104065 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:06.104065 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:06.104065 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:06.104065 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:06.104065 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:06.104065 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:01:06.104065 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:01:06.104065 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:01:06.104065 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:01:06.105607 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:06.105607 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:06.105607 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:06.105607 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:06.105607 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:06.105607 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:06.105607 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:06.105607 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:06.105607 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:06.105607 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:06.105607 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:06.105607 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:06.105607 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:06.105607 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:06.105607 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:06.105607 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:06.105607 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:01:06.107092 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:01:06.107092 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:01:06.107092 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:06.107092 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 443/866] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:01:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:01:06.113248 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a671483-e0de-49ba-8b37-b1011603433a None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:06.221897 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a671483-e0de-49ba-8b37-b1011603433a None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:06.258302 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a752f87b-d4d6-4378-8d53-f6521d6f150e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:06.258907 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752f87b-d4d6-4378-8d53-f6521d6f150e None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:06.259182 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752f87b-d4d6-4378-8d53-f6521d6f150e None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:06.259447 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752f87b-d4d6-4378-8d53-f6521d6f150e None admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:06.260085 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a752f87b-d4d6-4378-8d53-f6521d6f150e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:06.260409 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a752f87b-d4d6-4378-8d53-f6521d6f150e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:06.262619 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a752f87b-d4d6-4378-8d53-f6521d6f150e None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:06.269761 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:06.276169 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 424/867] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:01:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:06.282332 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc999b-b842-49ab-8ba4-6e9e70be98cd None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:06.305790 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:06.387734 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc999b-b842-49ab-8ba4-6e9e70be98cd None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:06.422265 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0190b974-e2dd-4b7a-935c-2436adb139c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:06.422887 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0190b974-e2dd-4b7a-935c-2436adb139c1 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:06.423162 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0190b974-e2dd-4b7a-935c-2436adb139c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:06.423433 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0190b974-e2dd-4b7a-935c-2436adb139c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:06.430295 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0190b974-e2dd-4b7a-935c-2436adb139c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:06.431747 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0190b974-e2dd-4b7a-935c-2436adb139c1 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:06.443838 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-0190b974-e2dd-4b7a-935c-2436adb139c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:01:06.443838 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:06.443838 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:06.443838 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:06.443838 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:06.443838 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:06.443838 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:06.443838 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:06.443838 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:06.443838 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:06.443838 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:06.443838 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:06.443838 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 01:01:06.443838 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 01:01:06.443838 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 01:01:06.444765 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 01:01:06.444765 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:06.444765 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:06.444765 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:06.444765 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:06.444765 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:06.444765 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:06.444765 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:06.444765 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:06.444765 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:06.444765 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:06.444765 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:06.444765 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:06.444765 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:06.444765 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:06.444765 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:06.444765 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:06.445652 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 01:01:06.445652 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 01:01:06.445652 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 01:01:06.445652 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 01:01:06.445652 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 01:01:06.445652 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 01:01:06.445652 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:01:06.445652 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:06.445652 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 444/868] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 01:01:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:01:06.453219 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a752f87b-d4d6-4378-8d53-f6521d6f150e None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:06.561050 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a752f87b-d4d6-4378-8d53-f6521d6f150e None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:06.595966 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698d1e6a-0918-4f11-b62f-f0c42d8c9ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:06.596604 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d1e6a-0918-4f11-b62f-f0c42d8c9ac1 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:06.596882 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d1e6a-0918-4f11-b62f-f0c42d8c9ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:06.597154 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d1e6a-0918-4f11-b62f-f0c42d8c9ac1 None admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:06.597846 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698d1e6a-0918-4f11-b62f-f0c42d8c9ac1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:06.598150 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698d1e6a-0918-4f11-b62f-f0c42d8c9ac1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:06.599587 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698d1e6a-0918-4f11-b62f-f0c42d8c9ac1 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:06.613242 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 425/869] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 8 01:01:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:06.619162 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0190b974-e2dd-4b7a-935c-2436adb139c1 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:06.717469 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:06.742960 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0190b974-e2dd-4b7a-935c-2436adb139c1 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:06.784856 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f143c-6315-4800-a47d-2204155c0fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:06.785847 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f143c-6315-4800-a47d-2204155c0fd5 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:06.786446 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f143c-6315-4800-a47d-2204155c0fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:06.786930 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f143c-6315-4800-a47d-2204155c0fd5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:06.787868 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8f143c-6315-4800-a47d-2204155c0fd5 None admin] RBAC: Adding query filter params (role.id=609d73f56e6a48698397131168abe6df, scope.project.id=78ad2a8fb5264fb4af935e039d28c8de, user.id=de75fd7d7721433684cafa2e6d3faaae) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:06.788588 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8f143c-6315-4800-a47d-2204155c0fd5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:06.790245 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8f143c-6315-4800-a47d-2204155c0fd5 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:06.797333 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 445/870] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 8 01:01:06 2022] GET /identity/v3/role_assignments?user.id=de75fd7d7721433684cafa2e6d3faaae&scope.project.id=78ad2a8fb5264fb4af935e039d28c8de&role.id=609d73f56e6a48698397131168abe6df => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:07.215256 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5d275-c31d-46db-be5a-ee15d05ca7a7 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:07.215818 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5d275-c31d-46db-be5a-ee15d05ca7a7 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:07.215818 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5d275-c31d-46db-be5a-ee15d05ca7a7 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:07.217407 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 426/871] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:01:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:01:07.222879 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2234bc05-5b62-4bcd-861a-40280f260e24 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:07.223165 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2234bc05-5b62-4bcd-861a-40280f260e24 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:07.223445 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2234bc05-5b62-4bcd-861a-40280f260e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:07.241838 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-2234bc05-5b62-4bcd-861a-40280f260e24 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:01:07.243032 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-2234bc05-5b62-4bcd-861a-40280f260e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:01:07.312935 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 446/872] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:01:07 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 01:01:07.580852 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a3d97c-e858-4a1c-89b1-d5669da98b55 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:07.581250 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a3d97c-e858-4a1c-89b1-d5669da98b55 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:07.581478 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a3d97c-e858-4a1c-89b1-d5669da98b55 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:07.582723 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2234bc05-5b62-4bcd-861a-40280f260e24 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:07.583300 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 427/873] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:01:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:01:07.589105 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98154cb4-10d2-4190-986a-ca6dfda972ed None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:07.589386 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98154cb4-10d2-4190-986a-ca6dfda972ed None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:07.589662 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98154cb4-10d2-4190-986a-ca6dfda972ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:07.609224 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-98154cb4-10d2-4190-986a-ca6dfda972ed None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:01:07.610435 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-98154cb4-10d2-4190-986a-ca6dfda972ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:01:07.666338 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:07.678872 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:07.696392 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2234bc05-5b62-4bcd-861a-40280f260e24 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:07.700245 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 428/874] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:01:07 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 01:01:07.733888 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb65d8ca-26af-45f1-a73f-a28774055076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:07.735182 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb65d8ca-26af-45f1-a73f-a28774055076 None admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:07.735674 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb65d8ca-26af-45f1-a73f-a28774055076 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:07.736181 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb65d8ca-26af-45f1-a73f-a28774055076 None admin] PATH_INFO: `/v3/services` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:07.737057 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb65d8ca-26af-45f1-a73f-a28774055076 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:07.738647 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb65d8ca-26af-45f1-a73f-a28774055076 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:07.755255 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 447/875] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Sep 8 01:01:07 2022] POST /identity/v3/services => generated 251 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:01:07.967623 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98154cb4-10d2-4190-986a-ca6dfda972ed None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:08.057173 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef37d01d-7c0f-4c7e-b77c-73a46cacc06d None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:08.057568 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef37d01d-7c0f-4c7e-b77c-73a46cacc06d None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:08.058285 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef37d01d-7c0f-4c7e-b77c-73a46cacc06d None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:08.060008 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 448/876] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:01:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 01:01:08.066316 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942057c4-10ea-4f2d-8d29-c912b0f7f38d None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:08.066698 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942057c4-10ea-4f2d-8d29-c912b0f7f38d None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:08.066968 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942057c4-10ea-4f2d-8d29-c912b0f7f38d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:08.074869 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98154cb4-10d2-4190-986a-ca6dfda972ed None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:08.097696 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-942057c4-10ea-4f2d-8d29-c912b0f7f38d None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:01:08.099905 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-942057c4-10ea-4f2d-8d29-c912b0f7f38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:01:08.116786 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1a0457-4ffc-4b55-bc50-ba24309768dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:08.117500 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a0457-4ffc-4b55-bc50-ba24309768dc None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:08.117846 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a0457-4ffc-4b55-bc50-ba24309768dc None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:08.118186 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a0457-4ffc-4b55-bc50-ba24309768dc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:08.118831 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1a0457-4ffc-4b55-bc50-ba24309768dc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:08.120372 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1a0457-4ffc-4b55-bc50-ba24309768dc None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:08.129863 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-cd1a0457-4ffc-4b55-bc50-ba24309768dc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 01:01:08.129863 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:08.129863 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 01:01:08.129863 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 01:01:08.129863 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 01:01:08.129863 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 01:01:08.129863 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 01:01:08.129863 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 01:01:08.129863 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 01:01:08.129863 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:08.129863 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 01:01:08.129863 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:08.129863 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:08.129863 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:08.129863 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:08.129863 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:08.129863 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:08.131031 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:08.131031 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:08.131031 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:08.131031 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:08.131031 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:08.131031 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:08.131031 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 01:01:08.131031 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 01:01:08.131031 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 01:01:08.131031 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 01:01:08.131031 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:08.131031 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:08.131031 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:08.131031 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:08.131031 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:08.131031 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:08.131951 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:08.131951 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:08.131951 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:08.131951 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:08.131951 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:08.131951 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:08.131951 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:08.131951 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:08.131951 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:08.131951 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:08.131951 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 01:01:08.131951 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 01:01:08.131951 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 01:01:08.131951 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:08.136771 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 429/877] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Sep 8 01:01:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:01:08.138539 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1a0457-4ffc-4b55-bc50-ba24309768dc None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:08.192030 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:08.202545 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 449/878] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:01:08 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 01:01:08.252900 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1a0457-4ffc-4b55-bc50-ba24309768dc None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:08.289252 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa508da2-dd1d-4ec7-8905-df2a1910f7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:08.289854 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa508da2-dd1d-4ec7-8905-df2a1910f7cf None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:08.290134 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa508da2-dd1d-4ec7-8905-df2a1910f7cf None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:08.290406 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa508da2-dd1d-4ec7-8905-df2a1910f7cf None admin] PATH_INFO: `/v3/services` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:08.291030 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa508da2-dd1d-4ec7-8905-df2a1910f7cf None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:08.291328 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa508da2-dd1d-4ec7-8905-df2a1910f7cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:08.292774 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa508da2-dd1d-4ec7-8905-df2a1910f7cf None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:08.298014 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 430/879] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Sep 8 01:01:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:01:08.310579 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942057c4-10ea-4f2d-8d29-c912b0f7f38d None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:08.374187 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:08.419322 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942057c4-10ea-4f2d-8d29-c912b0f7f38d None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:08.454368 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee09af-c2cd-40cd-8533-40f1e6dca55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:08.454977 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee09af-c2cd-40cd-8533-40f1e6dca55c None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:08.455258 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee09af-c2cd-40cd-8533-40f1e6dca55c None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:08.455532 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee09af-c2cd-40cd-8533-40f1e6dca55c None admin] PATH_INFO: `/v3/services` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:08.456177 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fee09af-c2cd-40cd-8533-40f1e6dca55c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:08.456487 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fee09af-c2cd-40cd-8533-40f1e6dca55c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:08.457850 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fee09af-c2cd-40cd-8533-40f1e6dca55c None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:08.462682 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 450/880] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Sep 8 01:01:08 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:08.484749 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa508da2-dd1d-4ec7-8905-df2a1910f7cf None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:08.600730 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa508da2-dd1d-4ec7-8905-df2a1910f7cf None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:08.615844 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:08.644447 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad6c697-4d61-45d0-9f47-d017d4aac21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:08.645068 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad6c697-4d61-45d0-9f47-d017d4aac21f None admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:08.645416 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad6c697-4d61-45d0-9f47-d017d4aac21f None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:08.645687 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad6c697-4d61-45d0-9f47-d017d4aac21f None admin] PATH_INFO: `/v3/services` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:08.646349 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad6c697-4d61-45d0-9f47-d017d4aac21f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:08.647799 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad6c697-4d61-45d0-9f47-d017d4aac21f None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:08.668542 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 431/881] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Sep 8 01:01:08 2022] POST /identity/v3/services => generated 269 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:01:09.166272 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a3730-92d5-4b12-a5d7-78bfc323167a None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:09.166871 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a3730-92d5-4b12-a5d7-78bfc323167a None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:09.167423 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a3730-92d5-4b12-a5d7-78bfc323167a None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:09.169275 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 451/882] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:01:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:01:09.174667 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ff95c-87d9-4fa7-a7d9-00665d4f96df None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:09.175062 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ff95c-87d9-4fa7-a7d9-00665d4f96df None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:09.175501 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ff95c-87d9-4fa7-a7d9-00665d4f96df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:09.196107 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-b23ff95c-87d9-4fa7-a7d9-00665d4f96df None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:01:09.197313 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-b23ff95c-87d9-4fa7-a7d9-00665d4f96df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:01:09.272478 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 432/883] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:01:09 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 01:01:09.560696 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766a3730-92d5-4b12-a5d7-78bfc323167a None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:09.675061 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766a3730-92d5-4b12-a5d7-78bfc323167a None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:09.710768 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74449750-2ec3-4178-9339-a668feb0e821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:09.711388 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74449750-2ec3-4178-9339-a668feb0e821 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:09.711660 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74449750-2ec3-4178-9339-a668feb0e821 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:09.711925 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74449750-2ec3-4178-9339-a668feb0e821 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:09.712620 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:09.723354 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74449750-2ec3-4178-9339-a668feb0e821 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:01:09.728405 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-74449750-2ec3-4178-9339-a668feb0e821 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 01:01:09.728405 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:09.728405 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:09.728405 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:09.728405 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:09.728405 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:09.728405 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:09.728405 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:09.728405 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:09.728405 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:09.728405 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:09.728405 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:09.728405 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:01:09.728405 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:01:09.728405 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:01:09.728405 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:01:09.729858 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:01:09.729858 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:01:09.729858 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:01:09.729858 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:01:09.729858 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:09.729858 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:09.729858 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:09.729858 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:09.729858 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:09.729858 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:09.729858 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:09.729858 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:09.729858 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:09.729858 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:09.729858 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:09.729858 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:09.730738 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:09.730738 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:09.730738 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:09.730738 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:09.730738 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:01:09.730738 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:01:09.730738 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:01:09.730738 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:01:09.730738 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:01:09.730738 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:01:09.730738 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 01:01:09.730738 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:09.730738 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 452/884] 10.222.0.42 () {56 vars in 1120 bytes} [Thu Sep 8 01:01:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:01:09.736548 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23ff95c-87d9-4fa7-a7d9-00665d4f96df None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:09.840278 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:09.844672 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23ff95c-87d9-4fa7-a7d9-00665d4f96df None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:09.887269 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67156b10-ed41-4d90-b526-e09ca53292ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:09.887878 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67156b10-ed41-4d90-b526-e09ca53292ed None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:09.888173 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67156b10-ed41-4d90-b526-e09ca53292ed None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:09.888452 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67156b10-ed41-4d90-b526-e09ca53292ed None admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:09.889106 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67156b10-ed41-4d90-b526-e09ca53292ed None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:09.889404 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67156b10-ed41-4d90-b526-e09ca53292ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:09.890767 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67156b10-ed41-4d90-b526-e09ca53292ed None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:09.899258 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 433/885] 10.222.0.42 () {56 vars in 1115 bytes} [Thu Sep 8 01:01:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:09.905976 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74449750-2ec3-4178-9339-a668feb0e821 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:10.018308 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74449750-2ec3-4178-9339-a668feb0e821 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:10.054249 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9f70cc-6423-46cc-8526-26dff17e6a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:10.054879 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f70cc-6423-46cc-8526-26dff17e6a9c None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:10.055151 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f70cc-6423-46cc-8526-26dff17e6a9c None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:10.055419 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f70cc-6423-46cc-8526-26dff17e6a9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:10.059961 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084453bd-a91c-40c8-b50c-6bb0604df204 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:10.060272 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084453bd-a91c-40c8-b50c-6bb0604df204 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:10.060547 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084453bd-a91c-40c8-b50c-6bb0604df204 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:10.062121 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 434/886] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:01:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:01:10.067848 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9f70cc-6423-46cc-8526-26dff17e6a9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:10.068199 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3d8f5b-0f1c-4dcb-b858-a3304d1b02c3 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:10.068493 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3d8f5b-0f1c-4dcb-b858-a3304d1b02c3 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:10.068770 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3d8f5b-0f1c-4dcb-b858-a3304d1b02c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:10.069228 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9f70cc-6423-46cc-8526-26dff17e6a9c None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:10.086707 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-dd9f70cc-6423-46cc-8526-26dff17e6a9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:01:10.086707 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:10.086707 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:10.086707 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:10.086707 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:10.086707 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:10.086707 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:10.086707 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:10.086707 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:10.086707 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:10.086707 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:10.086707 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:10.086707 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:01:10.086707 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:01:10.086707 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:01:10.086707 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:01:10.087793 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:10.087793 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:10.087793 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:10.087793 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:10.087793 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:10.087793 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:10.087793 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:10.087793 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:10.087793 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:10.087793 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:10.087793 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:10.087793 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:10.087793 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:10.087793 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:10.087793 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:10.087793 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:10.087793 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:01:10.089242 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:01:10.089242 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:01:10.089242 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:10.089433 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-2a3d8f5b-0f1c-4dcb-b858-a3304d1b02c3 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:01:10.089433 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-2a3d8f5b-0f1c-4dcb-b858-a3304d1b02c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:01:10.092280 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 453/887] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:01:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:01:10.094624 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9f70cc-6423-46cc-8526-26dff17e6a9c None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:10.143569 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:10.180342 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 435/888] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:01:10 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 01:01:10.220054 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9f70cc-6423-46cc-8526-26dff17e6a9c None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:10.258614 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783535c-5243-4182-bb64-93ad23c47e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:10.259277 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783535c-5243-4182-bb64-93ad23c47e42 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:10.259555 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783535c-5243-4182-bb64-93ad23c47e42 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:10.259874 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783535c-5243-4182-bb64-93ad23c47e42 None admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:10.260622 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2783535c-5243-4182-bb64-93ad23c47e42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:10.261897 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2783535c-5243-4182-bb64-93ad23c47e42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:10.263615 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2783535c-5243-4182-bb64-93ad23c47e42 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:10.275980 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 454/889] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:01:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:01:10.282036 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3d8f5b-0f1c-4dcb-b858-a3304d1b02c3 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:10.383791 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:10.399250 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3d8f5b-0f1c-4dcb-b858-a3304d1b02c3 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:10.436129 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f397bd8-290c-4848-bf8c-9c354924180f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:10.436779 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f397bd8-290c-4848-bf8c-9c354924180f None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:10.437057 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f397bd8-290c-4848-bf8c-9c354924180f None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:10.437325 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f397bd8-290c-4848-bf8c-9c354924180f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:10.444767 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f397bd8-290c-4848-bf8c-9c354924180f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:10.446208 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f397bd8-290c-4848-bf8c-9c354924180f None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:10.453206 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783535c-5243-4182-bb64-93ad23c47e42 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:10.460609 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-4f397bd8-290c-4848-bf8c-9c354924180f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 01:01:10.460609 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:10.460609 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:10.460609 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:10.460609 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:10.460609 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:10.460609 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:10.460609 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:10.460609 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:10.460609 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:10.460609 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:10.460609 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:10.460609 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 01:01:10.460609 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 01:01:10.460609 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 01:01:10.460609 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 01:01:10.461600 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:10.461600 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:10.461600 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 01:01:10.461600 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:01:10.461600 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 01:01:10.461600 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:01:10.461600 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:10.461600 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:10.461600 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:10.461600 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:10.461600 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:10.461600 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:10.461600 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:10.461600 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:10.461600 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:10.461600 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:10.461600 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:10.461600 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:10.462594 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:10.462594 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:10.462594 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 01:01:10.462594 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 01:01:10.462594 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 01:01:10.462594 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 01:01:10.462594 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 01:01:10.462594 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 01:01:10.462594 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 01:01:10.462594 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:10.462594 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 436/890] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Sep 8 01:01:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:01:10.468932 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f397bd8-290c-4848-bf8c-9c354924180f None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:10.566481 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:10.570254 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783535c-5243-4182-bb64-93ad23c47e42 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:10.611112 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f397bd8-290c-4848-bf8c-9c354924180f None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:10.615702 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d930ad-41e4-4e92-8062-75f726136da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:10.616823 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d930ad-41e4-4e92-8062-75f726136da9 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:10.617301 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d930ad-41e4-4e92-8062-75f726136da9 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:10.617794 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d930ad-41e4-4e92-8062-75f726136da9 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:10.618598 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d930ad-41e4-4e92-8062-75f726136da9 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:10.620561 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d930ad-41e4-4e92-8062-75f726136da9 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:10.632521 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-61d930ad-41e4-4e92-8062-75f726136da9 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 01:01:10.632521 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:10.632521 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 01:01:10.632521 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 01:01:10.632521 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 01:01:10.632521 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 01:01:10.632521 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 01:01:10.632521 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 01:01:10.632521 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 01:01:10.632521 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:10.632521 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 01:01:10.632521 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:10.632521 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:10.632521 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:10.632521 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:10.632521 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:10.632521 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:10.633629 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:10.633629 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:10.633629 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:10.633629 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:10.633629 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:10.633629 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:10.633629 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 01:01:10.633629 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 01:01:10.633629 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 01:01:10.633629 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 01:01:10.633629 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:10.633629 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:10.633629 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:10.633629 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:10.633629 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:10.633629 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:10.634493 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:10.634493 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:10.634493 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:10.634493 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:10.634493 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:10.634493 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:10.634493 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:10.634493 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:10.634493 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:10.634493 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:10.634493 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 01:01:10.634493 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 01:01:10.634493 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 01:01:10.634493 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:10.638343 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 455/891] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 8 01:01:10 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 01:01:10.645145 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d930ad-41e4-4e92-8062-75f726136da9 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:10.659141 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a0560-0057-4295-bf7c-dea0e340afb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:10.659915 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a0560-0057-4295-bf7c-dea0e340afb9 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:10.660205 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a0560-0057-4295-bf7c-dea0e340afb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:10.660484 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a0560-0057-4295-bf7c-dea0e340afb9 None admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:10.661222 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a0560-0057-4295-bf7c-dea0e340afb9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:10.661526 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a0560-0057-4295-bf7c-dea0e340afb9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:10.663142 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a0560-0057-4295-bf7c-dea0e340afb9 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:10.687860 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 437/892] 10.222.0.42 () {56 vars in 1149 bytes} [Thu Sep 8 01:01:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:10.688904 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a0560-0057-4295-bf7c-dea0e340afb9 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:10.700764 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:10.767941 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d930ad-41e4-4e92-8062-75f726136da9 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:10.798347 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a0560-0057-4295-bf7c-dea0e340afb9 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:10.803632 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bcdb53-194d-4b96-bb1f-51442a34f49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:10.804286 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bcdb53-194d-4b96-bb1f-51442a34f49e None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:10.804568 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bcdb53-194d-4b96-bb1f-51442a34f49e None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:10.804843 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bcdb53-194d-4b96-bb1f-51442a34f49e None admin] PATH_INFO: `/v3/services` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:10.805521 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bcdb53-194d-4b96-bb1f-51442a34f49e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:10.805826 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bcdb53-194d-4b96-bb1f-51442a34f49e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:10.807194 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bcdb53-194d-4b96-bb1f-51442a34f49e None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:10.817658 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 456/893] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 8 01:01:10 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:10.817658 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c754b9-d336-4b2d-bb5f-c22f9a4a4edc None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:10.817658 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c754b9-d336-4b2d-bb5f-c22f9a4a4edc None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:10.817658 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c754b9-d336-4b2d-bb5f-c22f9a4a4edc None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:10.817658 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 457/894] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:01:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:01:10.819576 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c754b9-d336-4b2d-bb5f-c22f9a4a4edc None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:10.843067 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e11393-4ed3-4990-9a38-6c89f148088e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:10.843676 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e11393-4ed3-4990-9a38-6c89f148088e None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:10.843954 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e11393-4ed3-4990-9a38-6c89f148088e None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:10.844250 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e11393-4ed3-4990-9a38-6c89f148088e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:10.852808 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:10.924606 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e11393-4ed3-4990-9a38-6c89f148088e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:10.925983 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e11393-4ed3-4990-9a38-6c89f148088e None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:10.939361 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:10.940777 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c754b9-d336-4b2d-bb5f-c22f9a4a4edc None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:10.979395 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da189cb-21d5-4a5b-85c2-0ada23075607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:10.980112 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da189cb-21d5-4a5b-85c2-0ada23075607 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:10.980417 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da189cb-21d5-4a5b-85c2-0ada23075607 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:10.980813 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da189cb-21d5-4a5b-85c2-0ada23075607 None admin] PATH_INFO: `/v3/services` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:10.981452 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da189cb-21d5-4a5b-85c2-0ada23075607 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:10.981929 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da189cb-21d5-4a5b-85c2-0ada23075607 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:10.983155 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da189cb-21d5-4a5b-85c2-0ada23075607 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:10.988322 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 458/895] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 8 01:01:10 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:01:10.989429 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9590e64e-bc9a-4d2c-8ef0-5b3a0038d572 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:10.989691 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9590e64e-bc9a-4d2c-8ef0-5b3a0038d572 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:10.989942 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9590e64e-bc9a-4d2c-8ef0-5b3a0038d572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:11.008092 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-9590e64e-bc9a-4d2c-8ef0-5b3a0038d572 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:01:11.009308 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-9590e64e-bc9a-4d2c-8ef0-5b3a0038d572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:01:11.036407 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 438/896] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 01:01:10 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 01:01:11.037749 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45cfb0-da8e-4484-a82f-97bc22baddab None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:11.038027 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45cfb0-da8e-4484-a82f-97bc22baddab None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:11.038301 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45cfb0-da8e-4484-a82f-97bc22baddab None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:11.039677 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 439/897] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:01:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:01:11.040293 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe45cfb0-da8e-4484-a82f-97bc22baddab None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:11.082087 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 459/898] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:01:10 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 01:01:11.082916 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9590e64e-bc9a-4d2c-8ef0-5b3a0038d572 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:11.107297 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:11.144539 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe45cfb0-da8e-4484-a82f-97bc22baddab None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:11.155921 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:11.185713 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63731da9-55c1-40ee-afe5-1dd72a1d70e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:11.186613 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63731da9-55c1-40ee-afe5-1dd72a1d70e5 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:11.186894 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63731da9-55c1-40ee-afe5-1dd72a1d70e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:11.187161 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63731da9-55c1-40ee-afe5-1dd72a1d70e5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:11.187863 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63731da9-55c1-40ee-afe5-1dd72a1d70e5 None admin] RBAC: Adding query filter params (interface=public, service_id=22304e52724f49b181ff168cc0859e50, region_id=RegionOne) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:11.188197 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63731da9-55c1-40ee-afe5-1dd72a1d70e5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:11.189569 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63731da9-55c1-40ee-afe5-1dd72a1d70e5 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:11.199120 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 440/899] 10.222.0.42 () {56 vars in 1251 bytes} [Thu Sep 8 01:01:11 2022] GET /identity/v3/endpoints?service_id=22304e52724f49b181ff168cc0859e50&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:11.199292 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9590e64e-bc9a-4d2c-8ef0-5b3a0038d572 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:11.199391 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ab4ac-9c9c-4fc2-b834-7866d759cedc None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:11.199391 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ab4ac-9c9c-4fc2-b834-7866d759cedc None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:11.199391 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ab4ac-9c9c-4fc2-b834-7866d759cedc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:11.215354 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-9b9ab4ac-9c9c-4fc2-b834-7866d759cedc None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:01:11.216604 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-9b9ab4ac-9c9c-4fc2-b834-7866d759cedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:01:11.234885 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f69fc5-d43d-4c6c-9b6a-4b6950f55ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:11.235495 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f69fc5-d43d-4c6c-9b6a-4b6950f55ab4 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:11.235770 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f69fc5-d43d-4c6c-9b6a-4b6950f55ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:11.236042 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f69fc5-d43d-4c6c-9b6a-4b6950f55ab4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:11.246481 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f69fc5-d43d-4c6c-9b6a-4b6950f55ab4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:11.247933 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f69fc5-d43d-4c6c-9b6a-4b6950f55ab4 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:11.264361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-80f69fc5-d43d-4c6c-9b6a-4b6950f55ab4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:01:11.264361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:11.264361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:11.264361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:11.264361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:11.264361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:11.264361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:11.264361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:11.264361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:11.264361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:11.264361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:11.264361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:11.264361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:01:11.264361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:01:11.264361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:01:11.264361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:01:11.265378 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:11.265378 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:11.265378 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:11.265378 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:11.265378 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:11.265378 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:11.265378 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:11.265378 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:11.265378 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:11.265378 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:11.265378 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:11.265378 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:11.265378 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:11.265378 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:11.265378 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:11.265378 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:11.265378 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:01:11.266292 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:01:11.266292 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:01:11.266292 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:11.266292 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 460/900] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:01:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:01:11.266292 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f69fc5-d43d-4c6c-9b6a-4b6950f55ab4 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:11.294059 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 441/901] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:01:11 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 01:01:11.294876 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ab4ac-9c9c-4fc2-b834-7866d759cedc None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:11.360165 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:11.388070 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f69fc5-d43d-4c6c-9b6a-4b6950f55ab4 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:11.410258 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ab4ac-9c9c-4fc2-b834-7866d759cedc None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:11.411752 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:11.427977 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceaf4aa-98e6-4090-bb42-0241f26b43dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:11.428895 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceaf4aa-98e6-4090-bb42-0241f26b43dc None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:11.429181 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceaf4aa-98e6-4090-bb42-0241f26b43dc None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:11.429452 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceaf4aa-98e6-4090-bb42-0241f26b43dc None admin] PATH_INFO: `/v3/services` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:11.430061 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceaf4aa-98e6-4090-bb42-0241f26b43dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:11.431832 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceaf4aa-98e6-4090-bb42-0241f26b43dc None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:11.448759 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 461/902] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Sep 8 01:01:11 2022] GET /identity/v3/services => generated 1268 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:01:11.448759 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceaf4aa-98e6-4090-bb42-0241f26b43dc None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:11.452853 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7557e567-0e67-4f1d-8413-f9537aa52d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:11.453485 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7557e567-0e67-4f1d-8413-f9537aa52d7e None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:11.453767 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7557e567-0e67-4f1d-8413-f9537aa52d7e None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:11.454043 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7557e567-0e67-4f1d-8413-f9537aa52d7e None admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:11.454712 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7557e567-0e67-4f1d-8413-f9537aa52d7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:11.455021 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7557e567-0e67-4f1d-8413-f9537aa52d7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:11.456477 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7557e567-0e67-4f1d-8413-f9537aa52d7e None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:11.473854 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 442/903] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:01:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:11.475556 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7557e567-0e67-4f1d-8413-f9537aa52d7e None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:11.575207 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceaf4aa-98e6-4090-bb42-0241f26b43dc None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:11.603650 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7557e567-0e67-4f1d-8413-f9537aa52d7e None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:11.613869 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511b321c-cb06-4957-a5cd-0260f16ed114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:11.614482 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b321c-cb06-4957-a5cd-0260f16ed114 None admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:11.614991 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b321c-cb06-4957-a5cd-0260f16ed114 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:11.615110 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b321c-cb06-4957-a5cd-0260f16ed114 None admin] PATH_INFO: `/v3/services` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:11.615678 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511b321c-cb06-4957-a5cd-0260f16ed114 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:11.619493 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511b321c-cb06-4957-a5cd-0260f16ed114 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:11.631778 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:11.640523 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 462/904] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Sep 8 01:01:11 2022] POST /identity/v3/services => generated 241 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:01:11.641658 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511b321c-cb06-4957-a5cd-0260f16ed114 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:11.670238 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c0c227-b880-427b-9023-75c96d4813e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:11.671341 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c0c227-b880-427b-9023-75c96d4813e2 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:11.671678 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c0c227-b880-427b-9023-75c96d4813e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:11.671959 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c0c227-b880-427b-9023-75c96d4813e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:11.683066 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c0c227-b880-427b-9023-75c96d4813e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:11.685132 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c0c227-b880-427b-9023-75c96d4813e2 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:11.702576 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-32c0c227-b880-427b-9023-75c96d4813e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:01:11.702576 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:11.702576 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:11.702576 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:11.702576 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:11.702576 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:11.702576 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:11.702576 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:11.702576 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:11.702576 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:11.702576 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:11.702576 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:11.702576 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 01:01:11.702576 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 01:01:11.702576 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 01:01:11.703691 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 01:01:11.703691 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:11.703691 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:11.703691 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:11.703691 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:11.703691 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:11.703691 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:11.703691 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:11.703691 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:11.703691 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:11.703691 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:11.703691 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:11.703691 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:11.703691 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:11.703691 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:11.703691 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:11.703691 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:11.704916 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 01:01:11.704916 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 01:01:11.704916 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 01:01:11.704916 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 01:01:11.704916 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 01:01:11.704916 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 01:01:11.704916 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:01:11.704916 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:11.704916 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 443/905] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 01:01:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:01:11.711707 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c0c227-b880-427b-9023-75c96d4813e2 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:11.741070 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:11.782732 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511b321c-cb06-4957-a5cd-0260f16ed114 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:11.831611 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0dfe0f-d638-4f72-b620-3e44937f7ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:11.831839 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0dfe0f-d638-4f72-b620-3e44937f7ea4 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:11.832115 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0dfe0f-d638-4f72-b620-3e44937f7ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:11.832313 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0dfe0f-d638-4f72-b620-3e44937f7ea4 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:11.832878 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0dfe0f-d638-4f72-b620-3e44937f7ea4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:11.835224 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0dfe0f-d638-4f72-b620-3e44937f7ea4 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:11.846116 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c0c227-b880-427b-9023-75c96d4813e2 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:11.847984 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-cf0dfe0f-d638-4f72-b620-3e44937f7ea4 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 01:01:11.847984 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:11.847984 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 01:01:11.847984 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 01:01:11.847984 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 01:01:11.847984 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 01:01:11.847984 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 01:01:11.847984 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 01:01:11.847984 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 01:01:11.847984 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:11.847984 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 01:01:11.847984 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:11.847984 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:11.847984 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:11.847984 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:11.847984 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:11.849054 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:11.849054 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:11.849054 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:11.849054 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:11.849054 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:11.849054 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:11.849054 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:11.849054 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 01:01:11.849054 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 01:01:11.849054 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 01:01:11.849054 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 01:01:11.849054 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:11.849054 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:11.849054 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:11.849054 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:11.849054 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:11.849054 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:11.849953 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:11.849953 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:11.849953 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:11.849953 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:11.849953 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:11.849953 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:11.849953 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:11.849953 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:11.849953 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:11.849953 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:11.849953 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 01:01:11.849953 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 01:01:11.849953 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 01:01:11.849953 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:11.849953 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 463/906] 10.222.0.42 () {56 vars in 1132 bytes} [Thu Sep 8 01:01:11 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:01:11.857353 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0dfe0f-d638-4f72-b620-3e44937f7ea4 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:11.890663 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c7e496-4efc-421e-b243-497bb649df66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:11.891280 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7e496-4efc-421e-b243-497bb649df66 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:11.891560 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7e496-4efc-421e-b243-497bb649df66 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:11.891831 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7e496-4efc-421e-b243-497bb649df66 None admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:11.892577 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c7e496-4efc-421e-b243-497bb649df66 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:11.892885 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c7e496-4efc-421e-b243-497bb649df66 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:11.894338 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c7e496-4efc-421e-b243-497bb649df66 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:11.907682 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 444/907] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 8 01:01:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:11.914942 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c7e496-4efc-421e-b243-497bb649df66 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:11.931521 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:11.980029 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0dfe0f-d638-4f72-b620-3e44937f7ea4 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:12.021233 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6a7dd2-e4c6-4218-a0f9-dd12165b0af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:12.022131 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a7dd2-e4c6-4218-a0f9-dd12165b0af5 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:12.022728 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a7dd2-e4c6-4218-a0f9-dd12165b0af5 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:12.023164 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a7dd2-e4c6-4218-a0f9-dd12165b0af5 None admin] PATH_INFO: `/v3/services` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:12.024320 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:12.025221 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6a7dd2-e4c6-4218-a0f9-dd12165b0af5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:12.025758 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6a7dd2-e4c6-4218-a0f9-dd12165b0af5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:12.027710 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6a7dd2-e4c6-4218-a0f9-dd12165b0af5 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:12.034793 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 464/908] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Sep 8 01:01:11 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:12.040745 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6a7dd2-e4c6-4218-a0f9-dd12165b0af5 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:12.053909 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c7e496-4efc-421e-b243-497bb649df66 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:12.094096 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07192388-2c72-4387-9206-bd6e2c534e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:12.094881 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07192388-2c72-4387-9206-bd6e2c534e7d None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:12.095272 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07192388-2c72-4387-9206-bd6e2c534e7d None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:12.095645 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07192388-2c72-4387-9206-bd6e2c534e7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:12.096512 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07192388-2c72-4387-9206-bd6e2c534e7d None admin] RBAC: Adding query filter params (role.id=62b04068ecbd4b13a405370cb199a142, scope.project.id=78ad2a8fb5264fb4af935e039d28c8de, user.id=de75fd7d7721433684cafa2e6d3faaae) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:12.096959 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07192388-2c72-4387-9206-bd6e2c534e7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:12.098717 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07192388-2c72-4387-9206-bd6e2c534e7d None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:12.112353 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 445/909] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 8 01:01:11 2022] GET /identity/v3/role_assignments?user.id=de75fd7d7721433684cafa2e6d3faaae&scope.project.id=78ad2a8fb5264fb4af935e039d28c8de&role.id=62b04068ecbd4b13a405370cb199a142 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:01:12.160037 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6a7dd2-e4c6-4218-a0f9-dd12165b0af5 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:12.190994 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:12.201882 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c8cdc7-2094-48fc-93d3-fd2970525c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:12.202508 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c8cdc7-2094-48fc-93d3-fd2970525c74 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:12.202788 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c8cdc7-2094-48fc-93d3-fd2970525c74 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:12.203061 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c8cdc7-2094-48fc-93d3-fd2970525c74 None admin] PATH_INFO: `/v3/services` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:12.203714 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c8cdc7-2094-48fc-93d3-fd2970525c74 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:12.204017 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c8cdc7-2094-48fc-93d3-fd2970525c74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:12.205529 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c8cdc7-2094-48fc-93d3-fd2970525c74 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:12.216118 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 465/910] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Sep 8 01:01:12 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:12.223047 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07192388-2c72-4387-9206-bd6e2c534e7d None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:12.338101 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07192388-2c72-4387-9206-bd6e2c534e7d None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:12.378244 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ce892d-8ab7-4985-a23a-0f487ab57bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:12.378873 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce892d-8ab7-4985-a23a-0f487ab57bbe None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:12.379154 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce892d-8ab7-4985-a23a-0f487ab57bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:12.379428 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce892d-8ab7-4985-a23a-0f487ab57bbe None admin] PATH_INFO: `/v3/endpoints` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:12.380159 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ce892d-8ab7-4985-a23a-0f487ab57bbe None admin] RBAC: Adding query filter params (interface=public, service_id=d5677fa0c69242649a40a4059252bfd1, region_id=RegionOne) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:12.380483 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ce892d-8ab7-4985-a23a-0f487ab57bbe None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:12.381883 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ce892d-8ab7-4985-a23a-0f487ab57bbe None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:12.388732 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 446/911] 10.222.0.42 () {56 vars in 1251 bytes} [Thu Sep 8 01:01:12 2022] GET /identity/v3/endpoints?service_id=d5677fa0c69242649a40a4059252bfd1&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:12.394278 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c8cdc7-2094-48fc-93d3-fd2970525c74 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:12.504857 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c8cdc7-2094-48fc-93d3-fd2970525c74 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:12.540067 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9256b8c-fd03-4ed3-852f-779369c2c9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:12.540708 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9256b8c-fd03-4ed3-852f-779369c2c9be None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:12.540986 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9256b8c-fd03-4ed3-852f-779369c2c9be None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:12.541257 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9256b8c-fd03-4ed3-852f-779369c2c9be None admin] PATH_INFO: `/v3/services` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:12.542384 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9256b8c-fd03-4ed3-852f-779369c2c9be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:12.543752 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9256b8c-fd03-4ed3-852f-779369c2c9be None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:12.550023 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 466/912] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Sep 8 01:01:12 2022] GET /identity/v3/services => generated 1497 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:01:13.673395 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecbe7d1-28aa-41c2-aeca-9016d1372a06 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:13.673941 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecbe7d1-28aa-41c2-aeca-9016d1372a06 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:13.673941 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecbe7d1-28aa-41c2-aeca-9016d1372a06 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:13.675402 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 447/913] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:01:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:01:13.681445 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e69b88-5a3b-49fa-a298-ee0b74b6a9ee None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:13.682309 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e69b88-5a3b-49fa-a298-ee0b74b6a9ee None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:13.683334 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e69b88-5a3b-49fa-a298-ee0b74b6a9ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:13.705049 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-82e69b88-5a3b-49fa-a298-ee0b74b6a9ee None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:01:13.706597 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-82e69b88-5a3b-49fa-a298-ee0b74b6a9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:01:13.761940 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:13.782653 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 467/914] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:01:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 01:01:13.905335 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c61514-9f40-4b80-a1b9-4becca305535 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:13.905561 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c61514-9f40-4b80-a1b9-4becca305535 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:13.905889 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c61514-9f40-4b80-a1b9-4becca305535 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:13.907360 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 448/915] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:01:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:01:13.913155 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd24d020-fee7-4da9-a08e-ddb48f82a351 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:13.913440 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd24d020-fee7-4da9-a08e-ddb48f82a351 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:13.913715 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd24d020-fee7-4da9-a08e-ddb48f82a351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:13.934336 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-bd24d020-fee7-4da9-a08e-ddb48f82a351 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:01:13.935593 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-bd24d020-fee7-4da9-a08e-ddb48f82a351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:01:14.018850 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 468/916] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:01:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 01:01:14.096187 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c61514-9f40-4b80-a1b9-4becca305535 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:14.106003 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:14.217030 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c61514-9f40-4b80-a1b9-4becca305535 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:14.254501 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5c95d0-0a48-414d-b0c7-14c0a84d2d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:14.255142 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c95d0-0a48-414d-b0c7-14c0a84d2d53 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:14.255432 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c95d0-0a48-414d-b0c7-14c0a84d2d53 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:14.255683 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c95d0-0a48-414d-b0c7-14c0a84d2d53 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:14.256301 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5c95d0-0a48-414d-b0c7-14c0a84d2d53 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:14.257690 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5c95d0-0a48-414d-b0c7-14c0a84d2d53 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:14.269016 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-6f5c95d0-0a48-414d-b0c7-14c0a84d2d53 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 01:01:14.269016 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:14.269016 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 01:01:14.269016 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 01:01:14.269016 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 01:01:14.269016 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 01:01:14.269016 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 01:01:14.269016 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 01:01:14.269016 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 01:01:14.269016 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:14.269016 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 01:01:14.269016 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:14.269016 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:14.269016 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:14.269016 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:14.269016 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:14.269016 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:14.270042 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:14.270042 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:14.270042 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:14.270042 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:14.270042 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:14.270042 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:14.270042 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 01:01:14.270042 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 01:01:14.270042 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 01:01:14.270042 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 01:01:14.270042 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:14.270042 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:14.270042 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:14.270042 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:14.270042 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:14.270042 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:14.270884 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:14.270884 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:14.270884 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:14.270884 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:14.270884 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:14.270884 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:14.270884 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:14.270884 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:14.270884 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:14.270884 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:14.270884 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 01:01:14.270884 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 01:01:14.270884 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 01:01:14.270884 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:14.270884 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 449/917] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 8 01:01:14 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:01:14.281618 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd24d020-fee7-4da9-a08e-ddb48f82a351 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:14.313469 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5c95d0-0a48-414d-b0c7-14c0a84d2d53 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:14.402196 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd24d020-fee7-4da9-a08e-ddb48f82a351 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:14.419102 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:14.426127 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:14.456191 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbd3a38-9a2d-4f09-9327-690db66ad969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:14.456840 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbd3a38-9a2d-4f09-9327-690db66ad969 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:14.457113 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbd3a38-9a2d-4f09-9327-690db66ad969 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:14.457373 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbd3a38-9a2d-4f09-9327-690db66ad969 None admin] PATH_INFO: `/v3/services` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:14.458422 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbd3a38-9a2d-4f09-9327-690db66ad969 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:14.458712 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbd3a38-9a2d-4f09-9327-690db66ad969 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:14.460409 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbd3a38-9a2d-4f09-9327-690db66ad969 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:14.468270 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 469/918] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 8 01:01:14 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:14.472530 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5c95d0-0a48-414d-b0c7-14c0a84d2d53 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:14.473450 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbd3a38-9a2d-4f09-9327-690db66ad969 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:14.512060 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84d2026-3b35-48cf-987c-e343d4b0b5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:14.512761 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d2026-3b35-48cf-987c-e343d4b0b5a7 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:14.513044 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d2026-3b35-48cf-987c-e343d4b0b5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:14.513316 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d2026-3b35-48cf-987c-e343d4b0b5a7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:14.513906 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84d2026-3b35-48cf-987c-e343d4b0b5a7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:14.515332 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84d2026-3b35-48cf-987c-e343d4b0b5a7 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:14.524639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-f84d2026-3b35-48cf-987c-e343d4b0b5a7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 01:01:14.524639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:14.524639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 01:01:14.524639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 01:01:14.524639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 01:01:14.524639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 01:01:14.524639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 01:01:14.524639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 01:01:14.524639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 01:01:14.524639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:14.524639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 01:01:14.524639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:14.524639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:14.524639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:14.524639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:14.524639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:14.524639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:14.525733 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:14.525733 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:14.525733 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:14.525733 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:14.525733 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:14.525733 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:14.525733 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 01:01:14.525733 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 01:01:14.525733 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 01:01:14.525733 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 01:01:14.525733 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:14.525733 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:14.525733 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:14.525733 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:14.525733 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:14.525733 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:14.527142 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:14.527142 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:14.527142 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:14.527142 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:14.527142 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:14.527142 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:14.527142 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:14.527142 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:14.527142 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:14.527142 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:14.527142 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 01:01:14.527142 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 01:01:14.527142 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 01:01:14.527142 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:14.527142 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 450/919] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Sep 8 01:01:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 01:01:14.528480 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54408132-18ae-4fad-8fb7-227219995974 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:14.528717 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54408132-18ae-4fad-8fb7-227219995974 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:14.529031 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54408132-18ae-4fad-8fb7-227219995974 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:14.530423 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 451/920] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:01:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:01:14.533468 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54408132-18ae-4fad-8fb7-227219995974 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:14.606599 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbd3a38-9a2d-4f09-9327-690db66ad969 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:14.652613 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f4fc05-ba98-43d6-bef8-654447fd2265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:14.653225 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4fc05-ba98-43d6-bef8-654447fd2265 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:14.653501 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4fc05-ba98-43d6-bef8-654447fd2265 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:14.653785 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4fc05-ba98-43d6-bef8-654447fd2265 None admin] PATH_INFO: `/v3/services` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:14.654468 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f4fc05-ba98-43d6-bef8-654447fd2265 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:14.654776 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f4fc05-ba98-43d6-bef8-654447fd2265 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:14.656166 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f4fc05-ba98-43d6-bef8-654447fd2265 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:14.661402 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 470/921] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Sep 8 01:01:14 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:01:14.662470 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f5d328-3d6d-4ec9-a9ec-8399487f30f8 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:14.662839 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f5d328-3d6d-4ec9-a9ec-8399487f30f8 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:14.663115 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f5d328-3d6d-4ec9-a9ec-8399487f30f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:14.677163 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54408132-18ae-4fad-8fb7-227219995974 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:14.681790 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-d8f5d328-3d6d-4ec9-a9ec-8399487f30f8 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:01:14.683087 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-d8f5d328-3d6d-4ec9-a9ec-8399487f30f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:01:14.713119 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b123bda-77a7-4b51-979d-5cfdff29b957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:14.714580 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b123bda-77a7-4b51-979d-5cfdff29b957 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:14.714905 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b123bda-77a7-4b51-979d-5cfdff29b957 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:14.715341 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b123bda-77a7-4b51-979d-5cfdff29b957 None admin] PATH_INFO: `/v3/services` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:14.715794 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b123bda-77a7-4b51-979d-5cfdff29b957 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:14.716095 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b123bda-77a7-4b51-979d-5cfdff29b957 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:14.717523 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b123bda-77a7-4b51-979d-5cfdff29b957 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:14.718692 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:14.722309 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:14.725989 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 452/922] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Sep 8 01:01:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:01:14.727062 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b123bda-77a7-4b51-979d-5cfdff29b957 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:14.765377 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 471/923] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:01:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 01:01:14.766532 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f5d328-3d6d-4ec9-a9ec-8399487f30f8 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:14.858343 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b123bda-77a7-4b51-979d-5cfdff29b957 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:14.883669 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f5d328-3d6d-4ec9-a9ec-8399487f30f8 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:14.906747 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9888b440-c813-409a-b84c-527ec2e511ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:14.908305 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9888b440-c813-409a-b84c-527ec2e511ec None admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:14.908618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9888b440-c813-409a-b84c-527ec2e511ec None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:14.908886 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9888b440-c813-409a-b84c-527ec2e511ec None admin] PATH_INFO: `/v3/endpoints` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:14.909565 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9888b440-c813-409a-b84c-527ec2e511ec None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:14.910956 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9888b440-c813-409a-b84c-527ec2e511ec None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:14.935955 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348cc385-e9c9-4fb8-a647-4dc5f870ce8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:14.936998 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348cc385-e9c9-4fb8-a647-4dc5f870ce8e None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:14.937533 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348cc385-e9c9-4fb8-a647-4dc5f870ce8e None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:14.938005 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348cc385-e9c9-4fb8-a647-4dc5f870ce8e None admin] PATH_INFO: `/v3/services` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:14.938569 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 453/924] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:01:14 2022] POST /identity/v3/endpoints => generated 346 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:01:14.939059 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348cc385-e9c9-4fb8-a647-4dc5f870ce8e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:14.939589 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348cc385-e9c9-4fb8-a647-4dc5f870ce8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:14.941358 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348cc385-e9c9-4fb8-a647-4dc5f870ce8e None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:14.950628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 472/925] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Sep 8 01:01:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:14.957384 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9888b440-c813-409a-b84c-527ec2e511ec None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:14.962023 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef15c45-db52-4588-8bc2-dffe2bb872f4 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:14.962330 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef15c45-db52-4588-8bc2-dffe2bb872f4 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:14.962583 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef15c45-db52-4588-8bc2-dffe2bb872f4 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:14.964086 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 473/926] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:01:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 01:01:14.969846 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c23aaf-b9fd-45f8-8ebb-2cf622a2d7cd None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:14.970119 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c23aaf-b9fd-45f8-8ebb-2cf622a2d7cd None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:14.970412 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c23aaf-b9fd-45f8-8ebb-2cf622a2d7cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:14.992326 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-24c23aaf-b9fd-45f8-8ebb-2cf622a2d7cd None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:01:14.993666 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-24c23aaf-b9fd-45f8-8ebb-2cf622a2d7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:01:15.029435 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:15.053098 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:15.088017 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 474/927] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:01:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 01:01:15.094056 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c23aaf-b9fd-45f8-8ebb-2cf622a2d7cd None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:15.110647 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9888b440-c813-409a-b84c-527ec2e511ec None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:15.153991 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09014559-5989-4cf9-8a4f-f809c88af5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:15.154605 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09014559-5989-4cf9-8a4f-f809c88af5ac None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:15.154885 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09014559-5989-4cf9-8a4f-f809c88af5ac None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:15.155157 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09014559-5989-4cf9-8a4f-f809c88af5ac None admin] PATH_INFO: `/v3/endpoints` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:15.155861 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09014559-5989-4cf9-8a4f-f809c88af5ac None admin] RBAC: Adding query filter params (interface=public, service_id=a5ff5a9dc3944aa3a6c46c8ad0e5e380, region_id=RegionOne) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:15.156233 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09014559-5989-4cf9-8a4f-f809c88af5ac None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:15.157673 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09014559-5989-4cf9-8a4f-f809c88af5ac None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:15.168161 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 454/928] 10.222.0.42 () {56 vars in 1251 bytes} [Thu Sep 8 01:01:14 2022] GET /identity/v3/endpoints?service_id=a5ff5a9dc3944aa3a6c46c8ad0e5e380&interface=public®ion_id=RegionOne => generated 194 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:15.170336 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09014559-5989-4cf9-8a4f-f809c88af5ac None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:15.228958 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c23aaf-b9fd-45f8-8ebb-2cf622a2d7cd None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:15.268411 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f65efb-4609-4e51-af68-c8e7186562af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:15.269090 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f65efb-4609-4e51-af68-c8e7186562af None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:15.269352 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f65efb-4609-4e51-af68-c8e7186562af None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:15.269625 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f65efb-4609-4e51-af68-c8e7186562af None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:15.281184 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f65efb-4609-4e51-af68-c8e7186562af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:01:15.285991 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-97f65efb-4609-4e51-af68-c8e7186562af None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 01:01:15.285991 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:15.285991 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:15.285991 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:15.285991 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:15.285991 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:15.285991 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:15.285991 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:15.285991 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:15.285991 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:15.285991 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:15.285991 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:15.285991 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:01:15.285991 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:01:15.285991 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:01:15.285991 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:01:15.286922 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:01:15.286922 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:01:15.286922 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:01:15.286922 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:01:15.286922 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:15.286922 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:15.286922 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:15.286922 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:15.286922 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:15.286922 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:15.286922 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:15.286922 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:15.286922 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:15.286922 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:15.286922 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:15.286922 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:15.287788 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:15.287788 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:15.287788 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:15.287788 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:15.287788 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:01:15.287788 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:01:15.287788 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:01:15.287788 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:01:15.287788 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:01:15.287788 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:01:15.287788 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 01:01:15.287788 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:15.292094 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 475/929] 10.222.0.42 () {56 vars in 1120 bytes} [Thu Sep 8 01:01:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 01:01:15.293414 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f65efb-4609-4e51-af68-c8e7186562af None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:15.305683 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09014559-5989-4cf9-8a4f-f809c88af5ac None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:15.319681 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:15.334989 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:15.354096 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4830030-3364-42b0-bf0e-20038dbee8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:15.354788 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4830030-3364-42b0-bf0e-20038dbee8d9 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:15.355093 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4830030-3364-42b0-bf0e-20038dbee8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:15.355380 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4830030-3364-42b0-bf0e-20038dbee8d9 None admin] PATH_INFO: `/v3/services` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:15.355993 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4830030-3364-42b0-bf0e-20038dbee8d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:15.357394 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4830030-3364-42b0-bf0e-20038dbee8d9 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:15.364182 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 455/930] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Sep 8 01:01:15 2022] GET /identity/v3/services => generated 1497 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 01:01:15.372189 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4830030-3364-42b0-bf0e-20038dbee8d9 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:15.446343 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f65efb-4609-4e51-af68-c8e7186562af None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:15.492573 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4830030-3364-42b0-bf0e-20038dbee8d9 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:15.493915 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f8991-25a9-4ae0-a791-339d5717d059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:15.494733 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f8991-25a9-4ae0-a791-339d5717d059 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:15.496111 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f8991-25a9-4ae0-a791-339d5717d059 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:15.496425 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f8991-25a9-4ae0-a791-339d5717d059 None admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:15.497122 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68f8991-25a9-4ae0-a791-339d5717d059 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:15.497435 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68f8991-25a9-4ae0-a791-339d5717d059 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:15.498852 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68f8991-25a9-4ae0-a791-339d5717d059 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:15.511868 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 476/931] 10.222.0.42 () {56 vars in 1115 bytes} [Thu Sep 8 01:01:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:01:15.518921 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f8991-25a9-4ae0-a791-339d5717d059 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:15.540849 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1573cf-953a-4308-9d89-ac9d39c03ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:15.541506 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1573cf-953a-4308-9d89-ac9d39c03ddc None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:15.542415 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1573cf-953a-4308-9d89-ac9d39c03ddc None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:15.542817 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1573cf-953a-4308-9d89-ac9d39c03ddc None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:15.543460 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1573cf-953a-4308-9d89-ac9d39c03ddc None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:15.545006 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1573cf-953a-4308-9d89-ac9d39c03ddc None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:15.554752 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-aa1573cf-953a-4308-9d89-ac9d39c03ddc None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 01:01:15.554752 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:15.554752 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 01:01:15.554752 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 01:01:15.554752 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 01:01:15.554752 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 01:01:15.554752 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 01:01:15.554752 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 01:01:15.554752 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 01:01:15.554752 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:15.554752 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 01:01:15.554752 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:15.554752 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:15.554752 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:15.554752 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:15.554752 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:15.555942 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:15.555942 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:15.555942 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:15.555942 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:15.555942 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:15.555942 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:15.555942 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:15.555942 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 01:01:15.555942 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 01:01:15.555942 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 01:01:15.555942 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 01:01:15.555942 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:15.555942 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:15.555942 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:15.555942 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:15.555942 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:15.555942 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:15.557024 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:15.557024 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:15.557024 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:15.557024 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:15.557024 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:15.557024 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:15.557024 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:15.557024 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:15.557024 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:15.557024 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:15.557024 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 01:01:15.557024 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 01:01:15.557024 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 01:01:15.557024 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:15.557024 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 456/932] 10.222.0.42 () {56 vars in 1132 bytes} [Thu Sep 8 01:01:15 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:01:15.563217 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1573cf-953a-4308-9d89-ac9d39c03ddc None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:15.620487 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:15.647515 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:15.666216 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f8991-25a9-4ae0-a791-339d5717d059 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:15.700954 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1573cf-953a-4308-9d89-ac9d39c03ddc None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:15.710726 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5746d3-77b1-400c-96d8-1dda6ebe4c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:15.711365 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5746d3-77b1-400c-96d8-1dda6ebe4c2d None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:15.712032 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5746d3-77b1-400c-96d8-1dda6ebe4c2d None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:15.712345 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5746d3-77b1-400c-96d8-1dda6ebe4c2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:15.727288 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5746d3-77b1-400c-96d8-1dda6ebe4c2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:15.730184 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5746d3-77b1-400c-96d8-1dda6ebe4c2d None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:15.737859 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209aadf5-824e-4afb-99cb-d25a1494ca6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:15.738498 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209aadf5-824e-4afb-99cb-d25a1494ca6a None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:15.738819 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209aadf5-824e-4afb-99cb-d25a1494ca6a None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:15.739110 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209aadf5-824e-4afb-99cb-d25a1494ca6a None admin] PATH_INFO: `/v3/services` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:15.739745 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209aadf5-824e-4afb-99cb-d25a1494ca6a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:15.740050 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209aadf5-824e-4afb-99cb-d25a1494ca6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:15.741472 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209aadf5-824e-4afb-99cb-d25a1494ca6a None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:15.748172 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-5f5746d3-77b1-400c-96d8-1dda6ebe4c2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:01:15.748172 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:15.748172 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:15.748172 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:15.748172 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:15.748172 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:15.748172 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:15.748172 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:15.748172 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:15.748172 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:15.748172 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:15.748172 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:15.748172 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:01:15.748172 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:01:15.748172 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:01:15.748172 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:01:15.749158 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:15.749158 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:15.749158 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:15.749158 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:15.749158 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:15.749158 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:15.749158 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:15.749158 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:15.749158 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:15.749158 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:15.749158 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:15.749158 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:15.749158 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:15.749158 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:15.749158 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:15.749158 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:15.749158 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:01:15.750088 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:01:15.750088 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:01:15.750088 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:15.751581 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 457/934] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Sep 8 01:01:15 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:01:15.752780 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209aadf5-824e-4afb-99cb-d25a1494ca6a None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:15.754159 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 477/934] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:01:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 01:01:15.755330 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5746d3-77b1-400c-96d8-1dda6ebe4c2d None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:15.861578 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209aadf5-824e-4afb-99cb-d25a1494ca6a None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:15.876606 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:15.892323 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5746d3-77b1-400c-96d8-1dda6ebe4c2d None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:15.911318 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccf8a53-785a-455e-a826-1020c7d36c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:15.911986 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf8a53-785a-455e-a826-1020c7d36c79 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:15.912443 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf8a53-785a-455e-a826-1020c7d36c79 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:15.912722 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf8a53-785a-455e-a826-1020c7d36c79 None admin] PATH_INFO: `/v3/services` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:15.913410 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccf8a53-785a-455e-a826-1020c7d36c79 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:15.913738 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccf8a53-785a-455e-a826-1020c7d36c79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:15.915974 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccf8a53-785a-455e-a826-1020c7d36c79 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:15.923582 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 458/935] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Sep 8 01:01:15 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:15.929873 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccf8a53-785a-455e-a826-1020c7d36c79 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:15.935746 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eb93dd-c4b7-43c3-9834-b2ad54750560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:15.936421 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb93dd-c4b7-43c3-9834-b2ad54750560 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:15.936703 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb93dd-c4b7-43c3-9834-b2ad54750560 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:15.936979 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb93dd-c4b7-43c3-9834-b2ad54750560 None admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:15.937759 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eb93dd-c4b7-43c3-9834-b2ad54750560 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:15.938096 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eb93dd-c4b7-43c3-9834-b2ad54750560 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:15.939527 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eb93dd-c4b7-43c3-9834-b2ad54750560 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:15.943255 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:15.957937 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 478/936] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:01:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:15.965201 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eb93dd-c4b7-43c3-9834-b2ad54750560 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:16.065478 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccf8a53-785a-455e-a826-1020c7d36c79 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:16.081880 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eb93dd-c4b7-43c3-9834-b2ad54750560 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:16.104846 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24431c74-c44d-4c64-82a7-efa0232bf50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:16.105697 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24431c74-c44d-4c64-82a7-efa0232bf50c None admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:16.106148 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24431c74-c44d-4c64-82a7-efa0232bf50c None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:16.106578 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24431c74-c44d-4c64-82a7-efa0232bf50c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:16.107382 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24431c74-c44d-4c64-82a7-efa0232bf50c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:16.108953 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24431c74-c44d-4c64-82a7-efa0232bf50c None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:16.118158 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5e46d8-647d-4a33-9549-dc08c881c809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:16.118850 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e46d8-647d-4a33-9549-dc08c881c809 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:16.119520 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e46d8-647d-4a33-9549-dc08c881c809 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:16.120311 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e46d8-647d-4a33-9549-dc08c881c809 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:16.134002 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 459/937] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:01:15 2022] POST /identity/v3/endpoints => generated 347 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 01:01:16.139204 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5e46d8-647d-4a33-9549-dc08c881c809 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:16.144179 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5e46d8-647d-4a33-9549-dc08c881c809 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:16.159723 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-2a5e46d8-647d-4a33-9549-dc08c881c809 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 01:01:16.159723 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:16.159723 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:16.159723 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:16.159723 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:16.159723 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:16.159723 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:16.159723 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:16.159723 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:16.159723 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:16.159723 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:16.159723 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:16.159723 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 01:01:16.159723 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 01:01:16.159723 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 01:01:16.159723 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 01:01:16.160966 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:16.160966 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:16.160966 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 01:01:16.160966 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:01:16.160966 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 01:01:16.160966 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:01:16.160966 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:16.160966 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:16.160966 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:16.160966 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:16.160966 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:16.160966 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:16.160966 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:16.160966 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:16.160966 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:16.160966 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:16.160966 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:16.160966 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:16.162117 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:16.162117 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:16.162117 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 01:01:16.162117 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 01:01:16.162117 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 01:01:16.162117 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 01:01:16.162117 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 01:01:16.162117 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 01:01:16.162117 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 01:01:16.162117 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:16.164485 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 479/938] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Sep 8 01:01:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 01:01:16.171340 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24431c74-c44d-4c64-82a7-efa0232bf50c None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:16.216558 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:16.311113 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24431c74-c44d-4c64-82a7-efa0232bf50c None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:16.352314 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb479b2-1c20-4e3c-9915-10f404c24f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:16.352845 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb479b2-1c20-4e3c-9915-10f404c24f40 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:16.353124 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb479b2-1c20-4e3c-9915-10f404c24f40 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:16.353399 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb479b2-1c20-4e3c-9915-10f404c24f40 None admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:16.354675 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb479b2-1c20-4e3c-9915-10f404c24f40 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:16.354996 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb479b2-1c20-4e3c-9915-10f404c24f40 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:16.358186 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb479b2-1c20-4e3c-9915-10f404c24f40 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:16.375901 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 460/939] 10.222.0.42 () {56 vars in 1149 bytes} [Thu Sep 8 01:01:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:16.381637 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5e46d8-647d-4a33-9549-dc08c881c809 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:16.441862 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:16.488397 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5e46d8-647d-4a33-9549-dc08c881c809 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:16.523115 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb8ad44-a10c-4910-8eef-4f8486ae3368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:16.523742 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8ad44-a10c-4910-8eef-4f8486ae3368 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:16.524023 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8ad44-a10c-4910-8eef-4f8486ae3368 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:16.524328 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8ad44-a10c-4910-8eef-4f8486ae3368 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:16.535216 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb8ad44-a10c-4910-8eef-4f8486ae3368 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:16.536708 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb8ad44-a10c-4910-8eef-4f8486ae3368 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:16.551684 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-fbb8ad44-a10c-4910-8eef-4f8486ae3368 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:01:16.551684 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:16.551684 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:16.551684 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:16.551684 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:16.551684 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:16.551684 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:16.551684 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:16.551684 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:16.551684 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:16.551684 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:16.551684 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:16.551684 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:01:16.551684 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:01:16.551684 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:01:16.551684 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:01:16.552708 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:16.552708 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:16.552708 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:16.552708 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:16.552708 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:16.552708 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:16.552708 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:16.552708 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:16.552708 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:16.552708 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:16.552708 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:16.552708 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:16.552708 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:16.552708 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:16.552708 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:16.552708 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:16.552708 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:01:16.553944 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:01:16.553944 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:01:16.553944 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:16.553944 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 480/940] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:01:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:01:16.560312 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb479b2-1c20-4e3c-9915-10f404c24f40 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:16.654832 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:16.673183 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb479b2-1c20-4e3c-9915-10f404c24f40 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:16.708650 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bc69a3-f2e4-4a81-ba19-50612b2d8312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:16.709269 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc69a3-f2e4-4a81-ba19-50612b2d8312 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:16.709527 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc69a3-f2e4-4a81-ba19-50612b2d8312 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:16.709802 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc69a3-f2e4-4a81-ba19-50612b2d8312 None admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:16.710453 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bc69a3-f2e4-4a81-ba19-50612b2d8312 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:16.710756 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bc69a3-f2e4-4a81-ba19-50612b2d8312 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:16.712172 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bc69a3-f2e4-4a81-ba19-50612b2d8312 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:16.725435 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 461/941] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:01:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:16.731570 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb8ad44-a10c-4910-8eef-4f8486ae3368 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:16.842832 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb8ad44-a10c-4910-8eef-4f8486ae3368 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:16.852458 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:16.879910 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8feb7d-4721-48f1-94ad-b77f3f208373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:16.880923 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8feb7d-4721-48f1-94ad-b77f3f208373 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:16.881401 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8feb7d-4721-48f1-94ad-b77f3f208373 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:16.881869 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8feb7d-4721-48f1-94ad-b77f3f208373 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:16.889554 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8feb7d-4721-48f1-94ad-b77f3f208373 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:16.891228 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8feb7d-4721-48f1-94ad-b77f3f208373 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:16.903773 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-2d8feb7d-4721-48f1-94ad-b77f3f208373 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:01:16.903773 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:16.903773 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:16.903773 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:16.903773 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:16.903773 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:16.903773 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:16.903773 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:16.903773 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:16.903773 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:16.903773 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:16.903773 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:16.903773 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 01:01:16.903773 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 01:01:16.903773 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 01:01:16.904724 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 01:01:16.904724 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:16.904724 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:16.904724 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:16.904724 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:16.904724 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:16.904724 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:16.904724 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:16.904724 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:16.904724 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:16.904724 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:16.904724 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:16.904724 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:16.904724 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:16.904724 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:16.904724 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:16.904724 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:16.905600 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 01:01:16.905600 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 01:01:16.905600 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 01:01:16.905600 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 01:01:16.905600 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 01:01:16.905600 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 01:01:16.905600 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:01:16.905600 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:16.912230 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 481/942] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 01:01:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:01:16.917476 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bc69a3-f2e4-4a81-ba19-50612b2d8312 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:17.025845 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bc69a3-f2e4-4a81-ba19-50612b2d8312 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:17.063791 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269a9197-2197-4302-9e5a-dfd5cab5af59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:17.064424 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a9197-2197-4302-9e5a-dfd5cab5af59 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:17.064705 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a9197-2197-4302-9e5a-dfd5cab5af59 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:17.064978 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a9197-2197-4302-9e5a-dfd5cab5af59 None admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:17.065671 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269a9197-2197-4302-9e5a-dfd5cab5af59 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:17.065975 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269a9197-2197-4302-9e5a-dfd5cab5af59 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:17.067414 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269a9197-2197-4302-9e5a-dfd5cab5af59 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:17.081080 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 462/943] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 8 01:01:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:17.087063 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8feb7d-4721-48f1-94ad-b77f3f208373 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:17.202180 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8feb7d-4721-48f1-94ad-b77f3f208373 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:17.237742 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bfe575-c9b9-4b5e-97df-f56f601a1f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:17.238428 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bfe575-c9b9-4b5e-97df-f56f601a1f28 None admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:17.238707 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bfe575-c9b9-4b5e-97df-f56f601a1f28 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:17.238983 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bfe575-c9b9-4b5e-97df-f56f601a1f28 None admin] PATH_INFO: `/v3/projects/78ad2a8fb5264fb4af935e039d28c8de/users/de75fd7d7721433684cafa2e6d3faaae/roles/62b04068ecbd4b13a405370cb199a142` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:17.241261 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bfe575-c9b9-4b5e-97df-f56f601a1f28 None admin] RBAC: Authorizing `identity:create_grant(project_id=78ad2a8fb5264fb4af935e039d28c8de, user_id=de75fd7d7721433684cafa2e6d3faaae, role_id=62b04068ecbd4b13a405370cb199a142)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:17.242744 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bfe575-c9b9-4b5e-97df-f56f601a1f28 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:17.261543 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 482/944] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 8 01:01:17 2022] PUT /identity/v3/projects/78ad2a8fb5264fb4af935e039d28c8de/users/de75fd7d7721433684cafa2e6d3faaae/roles/62b04068ecbd4b13a405370cb199a142 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:01:17.269896 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d62847-6994-4358-995a-b9dd7a604405 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:17.270201 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d62847-6994-4358-995a-b9dd7a604405 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:17.270480 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d62847-6994-4358-995a-b9dd7a604405 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:17.276218 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 463/945] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:01:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:01:17.278477 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784e956a-9897-4bfe-b0f0-dea23fca8b63 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:17.278768 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784e956a-9897-4bfe-b0f0-dea23fca8b63 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:17.279045 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784e956a-9897-4bfe-b0f0-dea23fca8b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:17.300685 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-784e956a-9897-4bfe-b0f0-dea23fca8b63 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:01:17.301929 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-784e956a-9897-4bfe-b0f0-dea23fca8b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:01:17.328978 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:17.379393 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 483/946] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:01:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 01:01:17.664253 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d62847-6994-4358-995a-b9dd7a604405 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:17.669802 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:17.690146 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574cac8c-dba0-4546-a906-8dbe64d262b0 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:17.690473 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574cac8c-dba0-4546-a906-8dbe64d262b0 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:17.690685 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574cac8c-dba0-4546-a906-8dbe64d262b0 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:17.693264 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 484/947] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:01:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:01:17.697290 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c373f172-8efe-429e-a388-e0865186b854 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:17.697570 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c373f172-8efe-429e-a388-e0865186b854 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:17.697874 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c373f172-8efe-429e-a388-e0865186b854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:17.717058 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-c373f172-8efe-429e-a388-e0865186b854 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:01:17.718352 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-c373f172-8efe-429e-a388-e0865186b854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:01:17.779403 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d62847-6994-4358-995a-b9dd7a604405 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:17.800398 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 485/948] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:01:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 01:01:17.817918 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5669e0-3eaa-446d-9b8d-c4d83687b997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:17.818581 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5669e0-3eaa-446d-9b8d-c4d83687b997 None admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:17.818859 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5669e0-3eaa-446d-9b8d-c4d83687b997 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:17.819127 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5669e0-3eaa-446d-9b8d-c4d83687b997 None admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:17.819818 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5669e0-3eaa-446d-9b8d-c4d83687b997 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:17.821348 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5669e0-3eaa-446d-9b8d-c4d83687b997 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:17.824632 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: WARNING keystone.server.flask.common [None req-2e5669e0-3eaa-446d-9b8d-c4d83687b997 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 08 01:01:17.842383 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.sql.core [None req-2e5669e0-3eaa-446d-9b8d-c4d83687b997 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 08 01:01:17.842383 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 08 01:01:17.842383 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [parameters: {'id': 'e4f5ea169c934d7ba55976fba26c7185', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 08 01:01:17.842383 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81661) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 08 01:01:17.850054 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-2e5669e0-3eaa-446d-9b8d-c4d83687b997 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e4f5ea169c934d7ba55976fba26c7185.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e4f5ea169c934d7ba55976fba26c7185. Sep 08 01:01:17.850054 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:17.850054 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 08 01:01:17.850054 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 08 01:01:17.850054 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 08 01:01:17.850054 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 08 01:01:17.850054 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 08 01:01:17.850054 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application result = self._query(query) Sep 08 01:01:17.850054 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 08 01:01:17.850054 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application conn.query(q) Sep 08 01:01:17.850054 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 08 01:01:17.850054 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 08 01:01:17.850054 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 08 01:01:17.850870 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application result.read() Sep 08 01:01:17.850870 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 08 01:01:17.850870 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 08 01:01:17.850870 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 08 01:01:17.850870 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 08 01:01:17.850870 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 08 01:01:17.850870 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 08 01:01:17.850870 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 08 01:01:17.850870 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 08 01:01:17.850870 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 08 01:01:17.850870 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:17.850870 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 08 01:01:17.850870 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:17.850870 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:17.850870 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 08 01:01:17.850870 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 08 01:01:17.850870 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 08 01:01:17.850870 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 08 01:01:17.851830 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 08 01:01:17.851830 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application next(self.gen) Sep 08 01:01:17.851830 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 08 01:01:17.851830 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application yield resource Sep 08 01:01:17.851830 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 08 01:01:17.851830 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application next(self.gen) Sep 08 01:01:17.851830 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 08 01:01:17.851830 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application self.session.rollback() Sep 08 01:01:17.851830 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 08 01:01:17.851830 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application self.force_reraise() Sep 08 01:01:17.851830 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 08 01:01:17.851830 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise self.value Sep 08 01:01:17.851830 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 08 01:01:17.851830 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 08 01:01:17.851830 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 08 01:01:17.851830 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application session.commit() Sep 08 01:01:17.851830 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 08 01:01:17.852751 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 08 01:01:17.852751 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 08 01:01:17.852751 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application self._prepare_impl() Sep 08 01:01:17.852751 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 08 01:01:17.852751 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application self.session.flush() Sep 08 01:01:17.852751 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 08 01:01:17.852751 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application self._flush(objects) Sep 08 01:01:17.852751 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 08 01:01:17.852751 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 08 01:01:17.852751 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 08 01:01:17.852751 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application compat.raise_( Sep 08 01:01:17.852751 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 08 01:01:17.852751 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception Sep 08 01:01:17.852751 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 08 01:01:17.852751 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application flush_context.execute() Sep 08 01:01:17.852751 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 08 01:01:17.852751 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rec.execute(self) Sep 08 01:01:17.853656 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 08 01:01:17.853656 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 08 01:01:17.853656 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 08 01:01:17.853656 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 08 01:01:17.853656 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 08 01:01:17.853656 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 08 01:01:17.853656 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 08 01:01:17.853656 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 08 01:01:17.853656 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 08 01:01:17.853656 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 08 01:01:17.853656 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 08 01:01:17.853656 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 08 01:01:17.853656 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 08 01:01:17.853656 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 08 01:01:17.853656 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 08 01:01:17.855016 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 08 01:01:17.855016 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 08 01:01:17.855016 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception Sep 08 01:01:17.855016 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 08 01:01:17.855016 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 08 01:01:17.855016 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 08 01:01:17.855016 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 08 01:01:17.855016 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 08 01:01:17.855016 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application result = self._query(query) Sep 08 01:01:17.855016 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 08 01:01:17.855016 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application conn.query(q) Sep 08 01:01:17.855016 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 08 01:01:17.855016 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 08 01:01:17.855016 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 08 01:01:17.855016 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application result.read() Sep 08 01:01:17.855016 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 08 01:01:17.855016 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 08 01:01:17.855882 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 08 01:01:17.855882 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 08 01:01:17.855882 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 08 01:01:17.855882 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 08 01:01:17.855882 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 08 01:01:17.855882 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 08 01:01:17.855882 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 08 01:01:17.855882 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 08 01:01:17.855882 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [parameters: {'id': 'e4f5ea169c934d7ba55976fba26c7185', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 08 01:01:17.855882 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 08 01:01:17.855882 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:17.855882 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 01:01:17.855882 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:17.855882 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:17.855882 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 08 01:01:17.858410 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 08 01:01:17.858410 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 08 01:01:17.858410 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 08 01:01:17.858410 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 08 01:01:17.858410 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:17.858410 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 01:01:17.858410 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:17.858410 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:17.858410 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:17.858410 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:17.858410 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:17.858410 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:17.858410 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:17.858410 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:17.858410 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:17.858410 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:17.858410 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:17.860525 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:17.860525 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 08 01:01:17.860525 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 08 01:01:17.860525 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:17.860525 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:17.860525 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 08 01:01:17.860525 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 08 01:01:17.860525 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e4f5ea169c934d7ba55976fba26c7185. Sep 08 01:01:17.860525 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:17.860525 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 464/949] 10.222.0.42 () {60 vars in 1140 bytes} [Thu Sep 8 01:01:17 2022] POST /identity/v3/projects => generated 262 bytes in 188 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:01:17.869819 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c373f172-8efe-429e-a388-e0865186b854 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:17.966155 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:17.980188 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c373f172-8efe-429e-a388-e0865186b854 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:18.021728 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ac5ddd-e985-4476-b6e0-326b1e0b1d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:18.022418 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ac5ddd-e985-4476-b6e0-326b1e0b1d17 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:18.022700 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ac5ddd-e985-4476-b6e0-326b1e0b1d17 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:18.022972 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ac5ddd-e985-4476-b6e0-326b1e0b1d17 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:18.030949 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ac5ddd-e985-4476-b6e0-326b1e0b1d17 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:18.032425 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ac5ddd-e985-4476-b6e0-326b1e0b1d17 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:18.045091 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-37ac5ddd-e985-4476-b6e0-326b1e0b1d17 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:01:18.045091 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:18.045091 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:18.045091 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:18.045091 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:18.045091 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:18.045091 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:18.045091 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:18.045091 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:18.045091 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:18.045091 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:18.045091 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:18.045091 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 01:01:18.045091 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 01:01:18.045091 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 01:01:18.046018 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 01:01:18.046018 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:18.046018 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:18.046018 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:18.046018 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:18.046018 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:18.046018 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:18.046018 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:18.046018 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:18.046018 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:18.046018 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:18.046018 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:18.046018 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:18.046018 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:18.046018 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:18.046018 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:18.046018 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:18.046892 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 01:01:18.046892 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 01:01:18.046892 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 01:01:18.046892 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 01:01:18.046892 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 01:01:18.046892 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 01:01:18.046892 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:01:18.046892 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:18.046892 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 486/950] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 01:01:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:01:18.058119 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5669e0-3eaa-446d-9b8d-c4d83687b997 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:18.094979 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ac5ddd-e985-4476-b6e0-326b1e0b1d17 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:18.140656 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:18.171784 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5669e0-3eaa-446d-9b8d-c4d83687b997 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:18.212267 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e28898-9e05-405e-84dc-fc16f8b336bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:18.212862 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ac5ddd-e985-4476-b6e0-326b1e0b1d17 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:18.214122 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e28898-9e05-405e-84dc-fc16f8b336bc None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:18.214606 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e28898-9e05-405e-84dc-fc16f8b336bc None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:18.215156 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e28898-9e05-405e-84dc-fc16f8b336bc None admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:18.216231 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e28898-9e05-405e-84dc-fc16f8b336bc None admin] RBAC: Adding query filter params (name=service) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:18.216925 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e28898-9e05-405e-84dc-fc16f8b336bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:18.218854 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e28898-9e05-405e-84dc-fc16f8b336bc None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:18.248004 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 465/951] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Sep 8 01:01:18 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:18.251069 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcc8d26-7c1b-4c23-ada7-6d4d42c424e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:18.251724 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc8d26-7c1b-4c23-ada7-6d4d42c424e0 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:18.251999 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc8d26-7c1b-4c23-ada7-6d4d42c424e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:18.252301 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc8d26-7c1b-4c23-ada7-6d4d42c424e0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:18.252895 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcc8d26-7c1b-4c23-ada7-6d4d42c424e0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:18.254279 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcc8d26-7c1b-4c23-ada7-6d4d42c424e0 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:18.265043 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-5fcc8d26-7c1b-4c23-ada7-6d4d42c424e0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 01:01:18.265043 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:18.265043 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 01:01:18.265043 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 01:01:18.265043 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 01:01:18.265043 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 01:01:18.265043 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 01:01:18.265043 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 01:01:18.265043 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 01:01:18.265043 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:18.265043 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 01:01:18.265043 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:18.265043 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:18.265043 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:18.265043 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:18.265043 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:18.265043 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:18.266119 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:18.266119 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:18.266119 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:18.266119 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:18.266119 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:18.266119 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:18.266119 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 01:01:18.266119 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 01:01:18.266119 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 01:01:18.266119 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 01:01:18.266119 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:18.266119 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:18.266119 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:18.266119 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:18.266119 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:18.266119 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:18.267151 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:18.267151 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:18.267151 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:18.267151 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:18.267151 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:18.267151 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:18.267151 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:18.267151 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:18.267151 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:18.267151 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:18.267151 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 01:01:18.267151 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 01:01:18.267151 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 01:01:18.267151 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:18.267151 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 487/952] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Sep 8 01:01:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 01:01:18.272640 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e28898-9e05-405e-84dc-fc16f8b336bc None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:18.385599 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e28898-9e05-405e-84dc-fc16f8b336bc None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:18.411120 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:18.434094 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bd5fcc-9b3b-434b-8b0f-c508432a7dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:18.435062 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd5fcc-9b3b-434b-8b0f-c508432a7dc4 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:18.435342 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd5fcc-9b3b-434b-8b0f-c508432a7dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:18.435607 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd5fcc-9b3b-434b-8b0f-c508432a7dc4 None admin] PATH_INFO: `/v3/services` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:18.436337 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bd5fcc-9b3b-434b-8b0f-c508432a7dc4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:18.436639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bd5fcc-9b3b-434b-8b0f-c508432a7dc4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:18.438033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bd5fcc-9b3b-434b-8b0f-c508432a7dc4 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:18.447773 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 466/953] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Sep 8 01:01:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:18.450536 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcc8d26-7c1b-4c23-ada7-6d4d42c424e0 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:18.469349 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:18.565643 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcc8d26-7c1b-4c23-ada7-6d4d42c424e0 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:18.588942 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c4e8f7-30ea-42e9-b017-7cbbc786d8fe None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:18.589290 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c4e8f7-30ea-42e9-b017-7cbbc786d8fe None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:18.589491 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c4e8f7-30ea-42e9-b017-7cbbc786d8fe None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:18.591126 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 467/954] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:01:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:01:18.596284 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28939657-a5dd-4aba-8095-e200249abcb9 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:18.596579 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28939657-a5dd-4aba-8095-e200249abcb9 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:18.596848 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28939657-a5dd-4aba-8095-e200249abcb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:18.605400 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adae667f-d2ba-45b9-8759-10e1ad026b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:18.605640 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adae667f-d2ba-45b9-8759-10e1ad026b06 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:18.606021 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adae667f-d2ba-45b9-8759-10e1ad026b06 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:18.606297 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adae667f-d2ba-45b9-8759-10e1ad026b06 None admin] PATH_INFO: `/v3/services` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:18.606940 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adae667f-d2ba-45b9-8759-10e1ad026b06 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:18.607248 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adae667f-d2ba-45b9-8759-10e1ad026b06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:18.608672 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adae667f-d2ba-45b9-8759-10e1ad026b06 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:18.614161 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 488/955] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Sep 8 01:01:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:18.622610 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adae667f-d2ba-45b9-8759-10e1ad026b06 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:18.625225 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-28939657-a5dd-4aba-8095-e200249abcb9 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:01:18.627004 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-28939657-a5dd-4aba-8095-e200249abcb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:01:18.707476 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:18.718291 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 468/956] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:01:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 01:01:18.737020 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adae667f-d2ba-45b9-8759-10e1ad026b06 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:18.773425 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0b6a3c-28f1-405e-a785-7595572c388c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:18.774024 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0b6a3c-28f1-405e-a785-7595572c388c None admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:18.774297 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0b6a3c-28f1-405e-a785-7595572c388c None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:18.774565 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0b6a3c-28f1-405e-a785-7595572c388c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:18.775225 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0b6a3c-28f1-405e-a785-7595572c388c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:18.777110 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0b6a3c-28f1-405e-a785-7595572c388c None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:18.798797 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 489/957] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:01:18 2022] POST /identity/v3/endpoints => generated 349 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:01:18.989144 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28939657-a5dd-4aba-8095-e200249abcb9 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:19.099693 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28939657-a5dd-4aba-8095-e200249abcb9 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:19.115141 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:19.143657 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e665c2e9-d222-427a-baa0-283e4e7c023d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:19.144323 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e665c2e9-d222-427a-baa0-283e4e7c023d None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:19.144603 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e665c2e9-d222-427a-baa0-283e4e7c023d None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:19.144869 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e665c2e9-d222-427a-baa0-283e4e7c023d None admin] PATH_INFO: `/v3/services/compute` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:19.145443 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e665c2e9-d222-427a-baa0-283e4e7c023d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:19.147095 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e665c2e9-d222-427a-baa0-283e4e7c023d None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:19.157789 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-e665c2e9-d222-427a-baa0-283e4e7c023d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 01:01:19.157789 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:19.157789 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 01:01:19.157789 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 01:01:19.157789 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 01:01:19.157789 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 01:01:19.157789 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 01:01:19.157789 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 01:01:19.157789 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 01:01:19.157789 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:19.157789 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 01:01:19.157789 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:19.157789 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:19.157789 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:19.157789 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:19.157789 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:19.157789 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:19.158920 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:19.158920 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:19.158920 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:19.158920 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:19.158920 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:19.158920 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:19.158920 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 01:01:19.158920 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 01:01:19.158920 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 01:01:19.158920 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 01:01:19.158920 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:19.158920 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:19.158920 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:19.158920 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:19.158920 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:19.158920 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:19.159772 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:19.159772 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:19.159772 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:19.159772 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:19.159772 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:19.159772 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:19.159772 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:19.159772 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:19.159772 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:19.159772 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:19.159772 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 01:01:19.159772 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 01:01:19.159772 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 01:01:19.159772 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:19.159772 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 469/958] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 01:01:18 2022] GET /identity/v3/services/compute => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:01:19.165015 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0b6a3c-28f1-405e-a785-7595572c388c None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:19.268561 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0b6a3c-28f1-405e-a785-7595572c388c None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:19.302262 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:19.306398 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1780acb4-90b5-41fc-8635-5fec21d677d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:19.307002 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1780acb4-90b5-41fc-8635-5fec21d677d8 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:19.307325 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1780acb4-90b5-41fc-8635-5fec21d677d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:19.308214 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1780acb4-90b5-41fc-8635-5fec21d677d8 None admin] PATH_INFO: `/v3/services` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:19.309163 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1780acb4-90b5-41fc-8635-5fec21d677d8 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:19.309639 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1780acb4-90b5-41fc-8635-5fec21d677d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:19.311174 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1780acb4-90b5-41fc-8635-5fec21d677d8 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:19.317262 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 490/959] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Sep 8 01:01:19 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:19.322836 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e665c2e9-d222-427a-baa0-283e4e7c023d None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:19.440000 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e665c2e9-d222-427a-baa0-283e4e7c023d None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:19.475851 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056cd820-bfdc-4373-8d95-ce2a7fc50628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:19.476487 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056cd820-bfdc-4373-8d95-ce2a7fc50628 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:19.476762 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056cd820-bfdc-4373-8d95-ce2a7fc50628 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:19.477033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056cd820-bfdc-4373-8d95-ce2a7fc50628 None admin] PATH_INFO: `/v3/services` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:19.477658 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056cd820-bfdc-4373-8d95-ce2a7fc50628 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:19.477958 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056cd820-bfdc-4373-8d95-ce2a7fc50628 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:19.480098 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056cd820-bfdc-4373-8d95-ce2a7fc50628 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:19.485373 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 470/960] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Sep 8 01:01:19 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:19.538493 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca61a8c-a45f-415e-9991-501555788694 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:19.538674 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca61a8c-a45f-415e-9991-501555788694 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:19.538946 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca61a8c-a45f-415e-9991-501555788694 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:19.540678 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 491/961] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:01:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:01:19.550167 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb9b28-4596-4a9b-9f4f-87650f4ca24b None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:19.550607 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb9b28-4596-4a9b-9f4f-87650f4ca24b None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:19.550884 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb9b28-4596-4a9b-9f4f-87650f4ca24b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:19.571134 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-cbfb9b28-4596-4a9b-9f4f-87650f4ca24b None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:01:19.572406 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-cbfb9b28-4596-4a9b-9f4f-87650f4ca24b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:01:19.609492 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:19.654648 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 471/962] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:01:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 01:01:19.927888 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca61a8c-a45f-415e-9991-501555788694 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:20.035245 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca61a8c-a45f-415e-9991-501555788694 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:20.071101 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b2acbb-cedd-4469-b01b-6290a22b6d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:20.071737 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b2acbb-cedd-4469-b01b-6290a22b6d21 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:20.072017 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b2acbb-cedd-4469-b01b-6290a22b6d21 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:20.072322 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b2acbb-cedd-4469-b01b-6290a22b6d21 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:20.083964 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b2acbb-cedd-4469-b01b-6290a22b6d21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:01:20.088881 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-69b2acbb-cedd-4469-b01b-6290a22b6d21 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 01:01:20.088881 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:20.088881 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:20.088881 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:20.088881 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:20.088881 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:20.088881 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:20.088881 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:20.088881 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:20.088881 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:20.088881 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:20.088881 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:20.088881 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:01:20.088881 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:01:20.088881 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:01:20.088881 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:01:20.089799 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:01:20.089799 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:01:20.089799 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:01:20.089799 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:01:20.089799 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:20.089799 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:20.089799 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:20.089799 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:20.089799 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:20.089799 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:20.089799 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:20.089799 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:20.089799 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:20.089799 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:20.089799 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:20.089799 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:20.091445 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:20.091445 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:20.091445 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:20.091445 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:20.091445 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:01:20.091445 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:01:20.091445 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:01:20.091445 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:01:20.091445 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:01:20.091445 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:01:20.091445 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 01:01:20.091445 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:20.091445 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 492/963] 10.222.0.42 () {56 vars in 1120 bytes} [Thu Sep 8 01:01:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:01:20.096481 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfb9b28-4596-4a9b-9f4f-87650f4ca24b None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:20.208012 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfb9b28-4596-4a9b-9f4f-87650f4ca24b None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:20.244414 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5166c70-4909-4bc1-859c-c6839060c3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:20.245375 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5166c70-4909-4bc1-859c-c6839060c3a8 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:20.245866 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5166c70-4909-4bc1-859c-c6839060c3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:20.246134 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5166c70-4909-4bc1-859c-c6839060c3a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:20.246802 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5166c70-4909-4bc1-859c-c6839060c3a8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:20.247100 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5166c70-4909-4bc1-859c-c6839060c3a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:20.248482 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5166c70-4909-4bc1-859c-c6839060c3a8 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:20.257442 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 472/964] 10.222.0.42 () {56 vars in 1115 bytes} [Thu Sep 8 01:01:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:20.263715 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b2acbb-cedd-4469-b01b-6290a22b6d21 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:20.327430 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:20.374369 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b2acbb-cedd-4469-b01b-6290a22b6d21 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:20.412363 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa21a3-dcd9-41e2-8426-cfc43c1d4065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:20.413005 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa21a3-dcd9-41e2-8426-cfc43c1d4065 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:20.413324 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa21a3-dcd9-41e2-8426-cfc43c1d4065 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:20.413658 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa21a3-dcd9-41e2-8426-cfc43c1d4065 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:20.424778 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fa21a3-dcd9-41e2-8426-cfc43c1d4065 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:20.426232 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fa21a3-dcd9-41e2-8426-cfc43c1d4065 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:20.441157 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-56fa21a3-dcd9-41e2-8426-cfc43c1d4065 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:01:20.441157 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:20.441157 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:20.441157 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:20.441157 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:20.441157 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:20.441157 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:20.441157 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:20.441157 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:20.441157 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:20.441157 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:20.441157 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:20.441157 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:01:20.441157 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:01:20.441157 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:01:20.441157 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:01:20.443392 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:20.443392 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:20.443392 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:20.443392 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:20.443392 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:20.443392 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:20.443392 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:20.443392 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:20.443392 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:20.443392 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:20.443392 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:20.443392 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:20.443392 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:20.443392 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:20.443392 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:20.443392 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:20.443392 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:01:20.445572 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:01:20.445572 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:01:20.445572 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:20.445572 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 493/965] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:01:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:01:20.448487 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5166c70-4909-4bc1-859c-c6839060c3a8 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:20.481561 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:20.555423 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5166c70-4909-4bc1-859c-c6839060c3a8 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:20.592550 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee51b087-1c0a-4bdd-8659-bd4dd8b4114a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:20.593224 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee51b087-1c0a-4bdd-8659-bd4dd8b4114a None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:20.593504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee51b087-1c0a-4bdd-8659-bd4dd8b4114a None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:20.593772 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee51b087-1c0a-4bdd-8659-bd4dd8b4114a None admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:20.594458 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee51b087-1c0a-4bdd-8659-bd4dd8b4114a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:20.594763 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee51b087-1c0a-4bdd-8659-bd4dd8b4114a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:20.596156 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee51b087-1c0a-4bdd-8659-bd4dd8b4114a None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:20.609198 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 473/966] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:01:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:20.617508 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa21a3-dcd9-41e2-8426-cfc43c1d4065 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:20.738367 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa21a3-dcd9-41e2-8426-cfc43c1d4065 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:20.748454 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:20.775029 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69b61c5-473d-46cf-97b2-cc3e5e1e49a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:20.775266 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2867bc2-1080-4661-ba07-39a45b860c18 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:20.775498 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2867bc2-1080-4661-ba07-39a45b860c18 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:20.775662 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2867bc2-1080-4661-ba07-39a45b860c18 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:20.775753 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b61c5-473d-46cf-97b2-cc3e5e1e49a6 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:20.775959 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b61c5-473d-46cf-97b2-cc3e5e1e49a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:20.776242 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b61c5-473d-46cf-97b2-cc3e5e1e49a6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:20.777651 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 474/967] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:01:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:01:20.782626 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f76abd5-c2b0-4940-8e4d-ca356b9f6b67 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:20.782909 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f76abd5-c2b0-4940-8e4d-ca356b9f6b67 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:20.783179 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f76abd5-c2b0-4940-8e4d-ca356b9f6b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:20.785261 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69b61c5-473d-46cf-97b2-cc3e5e1e49a6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:20.786833 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69b61c5-473d-46cf-97b2-cc3e5e1e49a6 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:20.799173 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-f69b61c5-473d-46cf-97b2-cc3e5e1e49a6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 01:01:20.799173 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:20.799173 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:20.799173 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:20.799173 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:20.799173 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:20.799173 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:20.799173 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:20.799173 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:20.799173 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:20.799173 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:20.799173 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:20.799173 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 01:01:20.799173 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 01:01:20.799173 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 01:01:20.799173 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 01:01:20.800194 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:20.800194 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:20.800194 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 01:01:20.800194 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:01:20.800194 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 01:01:20.800194 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:01:20.800194 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:20.800194 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:20.800194 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:20.800194 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:20.800194 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:20.800194 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:20.800194 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:20.800194 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:20.800194 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:20.800194 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:20.800194 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:20.800194 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:20.801140 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:20.801140 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:20.801140 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 01:01:20.801140 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 01:01:20.801140 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 01:01:20.801140 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 01:01:20.801140 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 01:01:20.801140 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 01:01:20.801140 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 01:01:20.801140 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:20.801140 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 494/968] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Sep 8 01:01:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:01:20.802392 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-9f76abd5-c2b0-4940-8e4d-ca356b9f6b67 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:01:20.803572 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-9f76abd5-c2b0-4940-8e4d-ca356b9f6b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:01:20.806504 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69b61c5-473d-46cf-97b2-cc3e5e1e49a6 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:20.884121 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 475/969] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:01:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 01:01:20.922203 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69b61c5-473d-46cf-97b2-cc3e5e1e49a6 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:20.958030 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf682847-418b-4dd2-9411-f9cc9d23a796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:20.958963 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf682847-418b-4dd2-9411-f9cc9d23a796 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:20.959407 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf682847-418b-4dd2-9411-f9cc9d23a796 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:20.959824 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf682847-418b-4dd2-9411-f9cc9d23a796 None admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:20.960720 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf682847-418b-4dd2-9411-f9cc9d23a796 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:20.961188 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf682847-418b-4dd2-9411-f9cc9d23a796 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:20.962804 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf682847-418b-4dd2-9411-f9cc9d23a796 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:20.984882 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 495/970] 10.222.0.42 () {56 vars in 1149 bytes} [Thu Sep 8 01:01:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:20.991092 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f76abd5-c2b0-4940-8e4d-ca356b9f6b67 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:21.016210 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:21.097716 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913266ae-dbb5-4f98-984f-7ba7b9a7b936 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:21.098015 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913266ae-dbb5-4f98-984f-7ba7b9a7b936 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:21.098260 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913266ae-dbb5-4f98-984f-7ba7b9a7b936 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:21.101007 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 496/971] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:01:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:01:21.102685 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f76abd5-c2b0-4940-8e4d-ca356b9f6b67 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:21.105753 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0771bb7-d2a9-4ee2-9e50-2f498beb3bab None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:21.106037 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0771bb7-d2a9-4ee2-9e50-2f498beb3bab None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:21.106304 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0771bb7-d2a9-4ee2-9e50-2f498beb3bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:21.125568 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-e0771bb7-d2a9-4ee2-9e50-2f498beb3bab None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:01:21.126760 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-e0771bb7-d2a9-4ee2-9e50-2f498beb3bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:01:21.139877 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e29026-3b11-4c1f-aa67-ac9a10ed7431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:21.140630 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e29026-3b11-4c1f-aa67-ac9a10ed7431 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:21.140913 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e29026-3b11-4c1f-aa67-ac9a10ed7431 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:21.141286 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e29026-3b11-4c1f-aa67-ac9a10ed7431 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:21.151623 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:21.210306 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 497/972] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:01:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 01:01:21.210971 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0771bb7-d2a9-4ee2-9e50-2f498beb3bab None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:21.225542 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e29026-3b11-4c1f-aa67-ac9a10ed7431 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:21.228277 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e29026-3b11-4c1f-aa67-ac9a10ed7431 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:21.256419 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:21.334120 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0771bb7-d2a9-4ee2-9e50-2f498beb3bab None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:21.351426 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 476/973] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 01:01:20 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 01:01:21.352534 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e29026-3b11-4c1f-aa67-ac9a10ed7431 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:21.382384 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762f1ed3-582f-456c-a048-cc6a28021739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:21.383351 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f1ed3-582f-456c-a048-cc6a28021739 None admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:21.383774 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f1ed3-582f-456c-a048-cc6a28021739 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:21.384310 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f1ed3-582f-456c-a048-cc6a28021739 None admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:21.385136 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762f1ed3-582f-456c-a048-cc6a28021739 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:21.386991 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762f1ed3-582f-456c-a048-cc6a28021739 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:21.390844 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: WARNING keystone.server.flask.common [None req-762f1ed3-582f-456c-a048-cc6a28021739 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 08 01:01:21.406175 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.sql.core [None req-762f1ed3-582f-456c-a048-cc6a28021739 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 08 01:01:21.406175 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 08 01:01:21.406175 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [parameters: {'user_id': '7476261a13974ff288b5341015218984', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 08 01:01:21.406175 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81660) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 08 01:01:21.430448 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-762f1ed3-582f-456c-a048-cc6a28021739 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 08 01:01:21.430448 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:21.430448 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 08 01:01:21.430448 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 08 01:01:21.430448 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 08 01:01:21.430448 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 08 01:01:21.430448 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 08 01:01:21.430448 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application result = self._query(query) Sep 08 01:01:21.430448 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 08 01:01:21.430448 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application conn.query(q) Sep 08 01:01:21.430448 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 08 01:01:21.430448 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 08 01:01:21.430448 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 08 01:01:21.430448 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application result.read() Sep 08 01:01:21.430448 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 08 01:01:21.431562 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 08 01:01:21.431562 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 08 01:01:21.431562 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 08 01:01:21.431562 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 08 01:01:21.431562 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 08 01:01:21.431562 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 08 01:01:21.431562 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 08 01:01:21.431562 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 08 01:01:21.431562 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:21.431562 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 08 01:01:21.431562 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:21.431562 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:21.431562 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 08 01:01:21.431562 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 08 01:01:21.431562 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 08 01:01:21.431562 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 08 01:01:21.431562 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 08 01:01:21.431562 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application next(self.gen) Sep 08 01:01:21.432796 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 08 01:01:21.432796 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application yield resource Sep 08 01:01:21.432796 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 08 01:01:21.432796 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application next(self.gen) Sep 08 01:01:21.432796 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 08 01:01:21.432796 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application self.session.rollback() Sep 08 01:01:21.432796 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 08 01:01:21.432796 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application self.force_reraise() Sep 08 01:01:21.432796 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 08 01:01:21.432796 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise self.value Sep 08 01:01:21.432796 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 08 01:01:21.432796 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 08 01:01:21.432796 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 08 01:01:21.432796 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application session.commit() Sep 08 01:01:21.432796 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 08 01:01:21.432796 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 08 01:01:21.432796 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 08 01:01:21.433958 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application self._prepare_impl() Sep 08 01:01:21.433958 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 08 01:01:21.433958 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application self.session.flush() Sep 08 01:01:21.433958 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 08 01:01:21.433958 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application self._flush(objects) Sep 08 01:01:21.433958 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 08 01:01:21.433958 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 08 01:01:21.433958 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 08 01:01:21.433958 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application compat.raise_( Sep 08 01:01:21.433958 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 08 01:01:21.433958 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception Sep 08 01:01:21.433958 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 08 01:01:21.433958 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application flush_context.execute() Sep 08 01:01:21.433958 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 08 01:01:21.433958 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rec.execute(self) Sep 08 01:01:21.433958 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 08 01:01:21.433958 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 08 01:01:21.435052 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 08 01:01:21.435052 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 08 01:01:21.435052 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 08 01:01:21.435052 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 08 01:01:21.435052 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 08 01:01:21.435052 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 08 01:01:21.435052 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 08 01:01:21.435052 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 08 01:01:21.435052 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 08 01:01:21.435052 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 08 01:01:21.435052 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 08 01:01:21.435052 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 08 01:01:21.435052 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 08 01:01:21.435052 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 08 01:01:21.435052 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 08 01:01:21.435052 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception Sep 08 01:01:21.436099 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 08 01:01:21.436099 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 08 01:01:21.436099 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 08 01:01:21.436099 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 08 01:01:21.436099 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 08 01:01:21.436099 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application result = self._query(query) Sep 08 01:01:21.436099 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 08 01:01:21.436099 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application conn.query(q) Sep 08 01:01:21.436099 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 08 01:01:21.436099 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 08 01:01:21.436099 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 08 01:01:21.436099 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application result.read() Sep 08 01:01:21.436099 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 08 01:01:21.436099 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 08 01:01:21.436099 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 08 01:01:21.436099 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 08 01:01:21.437171 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 08 01:01:21.437171 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 08 01:01:21.437171 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 08 01:01:21.437171 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 08 01:01:21.437171 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 08 01:01:21.437171 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 08 01:01:21.437171 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [parameters: {'user_id': '7476261a13974ff288b5341015218984', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 08 01:01:21.437171 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 08 01:01:21.437171 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:21.437171 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 01:01:21.437171 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:21.437171 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:21.437171 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:21.437171 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:21.437171 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:21.438160 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:21.438160 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:21.438160 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:21.438160 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:21.438160 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:21.438160 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:21.438160 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:21.438160 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 08 01:01:21.438160 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 08 01:01:21.438160 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:21.438160 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:21.438160 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 01:01:21.438160 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:01:21.438160 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 01:01:21.438160 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:01:21.438160 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 08 01:01:21.438160 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 08 01:01:21.439273 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 08 01:01:21.439273 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 08 01:01:21.439273 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 08 01:01:21.439273 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 08 01:01:21.439273 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 08 01:01:21.439273 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:21.441370 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 498/974] 10.222.0.42 () {60 vars in 1131 bytes} [Thu Sep 8 01:01:21 2022] POST /identity/v3/users => generated 160 bytes in 231 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:01:21.447566 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762f1ed3-582f-456c-a048-cc6a28021739 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:21.470328 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e29026-3b11-4c1f-aa67-ac9a10ed7431 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:21.475769 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:21.496121 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:21.512854 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07b337f-6a37-4195-8d6d-37c51dfe84c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:21.513407 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b337f-6a37-4195-8d6d-37c51dfe84c5 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:21.513691 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b337f-6a37-4195-8d6d-37c51dfe84c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:21.513954 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b337f-6a37-4195-8d6d-37c51dfe84c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:21.524510 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07b337f-6a37-4195-8d6d-37c51dfe84c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:21.526029 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07b337f-6a37-4195-8d6d-37c51dfe84c5 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:21.542341 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-b07b337f-6a37-4195-8d6d-37c51dfe84c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:01:21.542341 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:21.542341 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:21.542341 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:21.542341 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:21.542341 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:21.542341 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:21.542341 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:21.542341 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:21.542341 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:21.542341 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:21.542341 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:21.542341 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:01:21.542341 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:01:21.542341 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:01:21.542341 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:01:21.543507 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:21.543507 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:21.543507 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:21.543507 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:21.543507 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:21.543507 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:21.543507 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:21.543507 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:21.543507 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:21.543507 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:21.543507 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:21.543507 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:21.543507 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:21.543507 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:21.543507 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:21.543507 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:21.543507 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:01:21.544595 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:01:21.544595 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:01:21.544595 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:21.544595 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 477/975] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:01:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:01:21.544829 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07b337f-6a37-4195-8d6d-37c51dfe84c5 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:21.571776 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762f1ed3-582f-456c-a048-cc6a28021739 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:21.615581 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a685da23-7d0f-4208-9949-f36e8a9af90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:21.616269 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685da23-7d0f-4208-9949-f36e8a9af90c None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:21.616549 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685da23-7d0f-4208-9949-f36e8a9af90c None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:21.616818 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685da23-7d0f-4208-9949-f36e8a9af90c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:21.626517 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a685da23-7d0f-4208-9949-f36e8a9af90c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:21.628220 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a685da23-7d0f-4208-9949-f36e8a9af90c None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:21.645769 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-a685da23-7d0f-4208-9949-f36e8a9af90c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 01:01:21.645769 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:21.645769 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:21.645769 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:21.645769 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:21.645769 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:21.645769 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:21.645769 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:21.645769 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:21.645769 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:21.645769 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:21.645769 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:21.645769 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 01:01:21.645769 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 01:01:21.645769 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 01:01:21.645769 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 01:01:21.646830 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:21.646830 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:21.646830 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 01:01:21.646830 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:01:21.646830 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 01:01:21.646830 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:01:21.646830 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:21.646830 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:21.646830 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:21.646830 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:21.646830 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:21.646830 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:21.646830 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:21.646830 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:21.646830 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:21.646830 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:21.646830 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:21.646830 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:21.647775 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:21.647775 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:21.647775 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 01:01:21.647775 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 01:01:21.647775 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 01:01:21.647775 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 01:01:21.647775 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 01:01:21.647775 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 01:01:21.647775 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 01:01:21.647775 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:21.647775 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 499/976] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 01:01:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 01:01:21.648414 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a685da23-7d0f-4208-9949-f36e8a9af90c None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:21.670577 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07b337f-6a37-4195-8d6d-37c51dfe84c5 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:21.706596 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f538b05-451b-4d24-9dcc-609818322d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:21.707598 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f538b05-451b-4d24-9dcc-609818322d69 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:21.708071 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f538b05-451b-4d24-9dcc-609818322d69 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:21.708621 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f538b05-451b-4d24-9dcc-609818322d69 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:21.709207 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f538b05-451b-4d24-9dcc-609818322d69 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:21.710694 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f538b05-451b-4d24-9dcc-609818322d69 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:21.719661 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-6f538b05-451b-4d24-9dcc-609818322d69 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 01:01:21.719661 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:21.719661 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 01:01:21.719661 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 01:01:21.719661 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 01:01:21.719661 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 01:01:21.719661 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 01:01:21.719661 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 01:01:21.719661 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 01:01:21.719661 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:21.719661 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 01:01:21.719661 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:21.719661 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:21.719661 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:21.719661 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:21.719661 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:21.719661 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:21.721268 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:21.721268 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:21.721268 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:21.721268 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:21.721268 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:21.721268 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:21.721268 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 01:01:21.721268 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 01:01:21.721268 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 01:01:21.721268 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 01:01:21.721268 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:21.721268 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:21.721268 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:21.721268 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:21.721268 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:21.721268 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:21.722770 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:21.722770 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:21.722770 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:21.722770 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:21.722770 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:21.722770 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:21.722770 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:21.722770 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:21.722770 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:21.722770 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:21.722770 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 01:01:21.722770 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 01:01:21.722770 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 01:01:21.722770 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:21.722770 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 478/977] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Sep 8 01:01:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:01:21.723742 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f538b05-451b-4d24-9dcc-609818322d69 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:21.740715 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:21.760020 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:21.764167 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a685da23-7d0f-4208-9949-f36e8a9af90c None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:21.797588 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6992c6-5781-46d2-923c-8eaf8b35baaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:21.798181 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6992c6-5781-46d2-923c-8eaf8b35baaf None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:21.798455 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6992c6-5781-46d2-923c-8eaf8b35baaf None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:21.798721 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6992c6-5781-46d2-923c-8eaf8b35baaf None admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:21.799354 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6992c6-5781-46d2-923c-8eaf8b35baaf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:21.799654 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6992c6-5781-46d2-923c-8eaf8b35baaf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:21.801060 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6992c6-5781-46d2-923c-8eaf8b35baaf None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:21.816260 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 500/978] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:01:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:21.817536 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db9409-ad39-4e7a-bcfe-1c88841a4c8f None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:21.817820 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db9409-ad39-4e7a-bcfe-1c88841a4c8f None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:21.818087 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db9409-ad39-4e7a-bcfe-1c88841a4c8f None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:21.819474 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 501/979] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:01:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:01:21.819966 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db9409-ad39-4e7a-bcfe-1c88841a4c8f None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:21.830274 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f538b05-451b-4d24-9dcc-609818322d69 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:21.864535 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bfcebf-1c47-4f92-bdc9-fd44371532e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:21.865136 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bfcebf-1c47-4f92-bdc9-fd44371532e9 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:21.865417 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bfcebf-1c47-4f92-bdc9-fd44371532e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:21.865696 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bfcebf-1c47-4f92-bdc9-fd44371532e9 None admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:21.866391 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bfcebf-1c47-4f92-bdc9-fd44371532e9 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:21.866776 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bfcebf-1c47-4f92-bdc9-fd44371532e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:21.868242 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bfcebf-1c47-4f92-bdc9-fd44371532e9 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:21.903661 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 479/980] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Sep 8 01:01:21 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:21.904361 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bfcebf-1c47-4f92-bdc9-fd44371532e9 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:21.937473 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db9409-ad39-4e7a-bcfe-1c88841a4c8f None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:21.982685 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d5c1d-1a07-40f0-8cc0-17952bb97947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:21.983315 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d5c1d-1a07-40f0-8cc0-17952bb97947 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:21.983603 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d5c1d-1a07-40f0-8cc0-17952bb97947 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:21.983871 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d5c1d-1a07-40f0-8cc0-17952bb97947 None admin] PATH_INFO: `/v3/services` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:21.984608 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88d5c1d-1a07-40f0-8cc0-17952bb97947 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:21.984917 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88d5c1d-1a07-40f0-8cc0-17952bb97947 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:21.986383 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88d5c1d-1a07-40f0-8cc0-17952bb97947 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:21.991885 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 502/981] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Sep 8 01:01:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:21.993210 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c660d74-1339-4a5a-9816-1ae080b61ede None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:21.993496 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c660d74-1339-4a5a-9816-1ae080b61ede None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:21.993768 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c660d74-1339-4a5a-9816-1ae080b61ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:22.000982 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:22.015943 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-8c660d74-1339-4a5a-9816-1ae080b61ede None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:01:22.016529 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:22.018308 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-8c660d74-1339-4a5a-9816-1ae080b61ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:01:22.024642 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bfcebf-1c47-4f92-bdc9-fd44371532e9 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:22.061866 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a4e79e-d50e-430d-bed6-50b96f8c8eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:22.062946 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a4e79e-d50e-430d-bed6-50b96f8c8eb2 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:22.063229 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a4e79e-d50e-430d-bed6-50b96f8c8eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:22.063495 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a4e79e-d50e-430d-bed6-50b96f8c8eb2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:22.071407 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a4e79e-d50e-430d-bed6-50b96f8c8eb2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:22.073534 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a4e79e-d50e-430d-bed6-50b96f8c8eb2 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:22.086598 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-17a4e79e-d50e-430d-bed6-50b96f8c8eb2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:01:22.086598 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:22.086598 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:22.086598 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:22.086598 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:22.086598 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:22.086598 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:22.086598 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:22.086598 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:22.086598 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:22.086598 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:22.086598 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:22.086598 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 01:01:22.086598 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 01:01:22.086598 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 01:01:22.087663 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 01:01:22.087663 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:22.087663 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:22.087663 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:22.087663 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:22.087663 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:22.087663 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:22.087663 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:22.087663 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:22.087663 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:22.087663 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:22.087663 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:22.087663 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:22.087663 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:22.087663 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:22.087663 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:22.087663 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:22.089098 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 01:01:22.089098 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 01:01:22.089098 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 01:01:22.089098 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 01:01:22.089098 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 01:01:22.089098 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 01:01:22.089098 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:01:22.089098 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:22.089098 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 480/982] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 01:01:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:01:22.089717 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a4e79e-d50e-430d-bed6-50b96f8c8eb2 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:22.103626 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 503/983] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:01:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 01:01:22.104615 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c660d74-1339-4a5a-9816-1ae080b61ede None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:22.224199 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a4e79e-d50e-430d-bed6-50b96f8c8eb2 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:22.233847 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c660d74-1339-4a5a-9816-1ae080b61ede None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:22.259639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d28edd-3996-4295-a089-102da972f9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:22.260360 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d28edd-3996-4295-a089-102da972f9a2 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:22.260644 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d28edd-3996-4295-a089-102da972f9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:22.261111 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d28edd-3996-4295-a089-102da972f9a2 None admin] PATH_INFO: `/v3/services` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:22.262005 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d28edd-3996-4295-a089-102da972f9a2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:22.262315 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d28edd-3996-4295-a089-102da972f9a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:22.263788 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d28edd-3996-4295-a089-102da972f9a2 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:22.265607 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:22.274149 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 481/984] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Sep 8 01:01:22 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:22.274292 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdac4f4-4913-4e05-ad95-120bfcdc2926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:22.274292 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdac4f4-4913-4e05-ad95-120bfcdc2926 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:22.274292 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdac4f4-4913-4e05-ad95-120bfcdc2926 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:22.274292 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdac4f4-4913-4e05-ad95-120bfcdc2926 None admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:22.274586 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdac4f4-4913-4e05-ad95-120bfcdc2926 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:22.274586 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdac4f4-4913-4e05-ad95-120bfcdc2926 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:22.275082 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d28edd-3996-4295-a089-102da972f9a2 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:22.275907 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdac4f4-4913-4e05-ad95-120bfcdc2926 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:22.289497 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 504/985] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 8 01:01:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:22.295831 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdac4f4-4913-4e05-ad95-120bfcdc2926 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:22.296654 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:22.381093 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d28edd-3996-4295-a089-102da972f9a2 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:22.402363 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdac4f4-4913-4e05-ad95-120bfcdc2926 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:22.422867 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7201d96f-6869-486e-bdd1-e02b669eda77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:22.423478 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201d96f-6869-486e-bdd1-e02b669eda77 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:22.423761 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201d96f-6869-486e-bdd1-e02b669eda77 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:22.424037 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201d96f-6869-486e-bdd1-e02b669eda77 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:22.424777 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7201d96f-6869-486e-bdd1-e02b669eda77 None admin] RBAC: Adding query filter params (interface=admin, service_id=a5ff5a9dc3944aa3a6c46c8ad0e5e380, region_id=RegionOne) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:22.425090 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7201d96f-6869-486e-bdd1-e02b669eda77 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:22.426457 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7201d96f-6869-486e-bdd1-e02b669eda77 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:22.431411 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 482/986] 10.222.0.42 () {56 vars in 1249 bytes} [Thu Sep 8 01:01:22 2022] GET /identity/v3/endpoints?service_id=a5ff5a9dc3944aa3a6c46c8ad0e5e380&interface=admin®ion_id=RegionOne => generated 193 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:22.432207 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7201d96f-6869-486e-bdd1-e02b669eda77 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:22.439156 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9025c1a-f3b9-4b4b-9799-ad22301f8ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:22.439759 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9025c1a-f3b9-4b4b-9799-ad22301f8ad3 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:22.440038 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9025c1a-f3b9-4b4b-9799-ad22301f8ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:22.440354 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9025c1a-f3b9-4b4b-9799-ad22301f8ad3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:22.441089 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9025c1a-f3b9-4b4b-9799-ad22301f8ad3 None admin] RBAC: Adding query filter params (role.id=62b04068ecbd4b13a405370cb199a142, scope.project.id=78ad2a8fb5264fb4af935e039d28c8de, user.id=de75fd7d7721433684cafa2e6d3faaae) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:22.441397 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9025c1a-f3b9-4b4b-9799-ad22301f8ad3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:22.442811 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9025c1a-f3b9-4b4b-9799-ad22301f8ad3 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:22.449439 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 505/987] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 8 01:01:22 2022] GET /identity/v3/role_assignments?user.id=de75fd7d7721433684cafa2e6d3faaae&scope.project.id=78ad2a8fb5264fb4af935e039d28c8de&role.id=62b04068ecbd4b13a405370cb199a142 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:01:22.450209 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9025c1a-f3b9-4b4b-9799-ad22301f8ad3 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:22.521284 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:22.525085 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:22.544625 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7201d96f-6869-486e-bdd1-e02b669eda77 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:22.575525 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9025c1a-f3b9-4b4b-9799-ad22301f8ad3 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:22.585525 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f98d78-4410-4561-9576-1eb268694309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:22.586241 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f98d78-4410-4561-9576-1eb268694309 None admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:22.586528 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f98d78-4410-4561-9576-1eb268694309 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:22.586804 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f98d78-4410-4561-9576-1eb268694309 None admin] PATH_INFO: `/v3/services` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:22.587585 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f98d78-4410-4561-9576-1eb268694309 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:22.589840 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f98d78-4410-4561-9576-1eb268694309 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:22.612193 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 483/988] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Sep 8 01:01:22 2022] POST /identity/v3/services => generated 242 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:01:22.627499 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e27502-8d3d-42be-b27c-518bae63da14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:22.628228 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e27502-8d3d-42be-b27c-518bae63da14 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:22.628449 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e27502-8d3d-42be-b27c-518bae63da14 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:22.628720 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e27502-8d3d-42be-b27c-518bae63da14 None admin] PATH_INFO: `/v3/services` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:22.630705 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e27502-8d3d-42be-b27c-518bae63da14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:22.630840 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e27502-8d3d-42be-b27c-518bae63da14 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:22.639403 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 506/989] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Sep 8 01:01:22 2022] GET /identity/v3/services => generated 1727 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:01:24.681113 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ee986-e0e6-43c9-a0cc-bce7516f2e21 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:24.681685 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ee986-e0e6-43c9-a0cc-bce7516f2e21 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:24.681685 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ee986-e0e6-43c9-a0cc-bce7516f2e21 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:24.683708 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 484/990] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:01:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:01:24.689129 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b878fffc-1f40-45d9-9b52-43fe5c089ea5 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:24.689593 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b878fffc-1f40-45d9-9b52-43fe5c089ea5 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:24.689681 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b878fffc-1f40-45d9-9b52-43fe5c089ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:24.709384 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-b878fffc-1f40-45d9-9b52-43fe5c089ea5 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:01:24.710667 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-b878fffc-1f40-45d9-9b52-43fe5c089ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:01:24.786570 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 507/991] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:01:24 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 01:01:24.830770 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e764a37-6c74-403e-b51a-764349b3bb41 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:24.831083 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e764a37-6c74-403e-b51a-764349b3bb41 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:24.831321 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e764a37-6c74-403e-b51a-764349b3bb41 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:24.832828 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 485/992] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:01:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:01:24.834860 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583bdea1-cc3c-43a8-95f7-ad4979c5854b None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:24.835149 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583bdea1-cc3c-43a8-95f7-ad4979c5854b None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:24.835423 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583bdea1-cc3c-43a8-95f7-ad4979c5854b None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:24.836894 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 508/993] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:01:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:01:24.837718 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c1778-a99d-448d-b7fc-980a7dc45093 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:24.838056 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c1778-a99d-448d-b7fc-980a7dc45093 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:24.838385 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c1778-a99d-448d-b7fc-980a7dc45093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:24.841738 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1236d0-c771-4387-8a65-9374672a31d4 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:24.842031 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1236d0-c771-4387-8a65-9374672a31d4 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:24.842309 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1236d0-c771-4387-8a65-9374672a31d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:24.858047 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-e83c1778-a99d-448d-b7fc-980a7dc45093 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:01:24.859581 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-e83c1778-a99d-448d-b7fc-980a7dc45093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:01:24.863199 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-bc1236d0-c771-4387-8a65-9374672a31d4 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:01:24.864406 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-bc1236d0-c771-4387-8a65-9374672a31d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:01:24.910745 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:24.930623 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 486/994] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:01:24 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 01:01:24.939202 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 509/995] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:01:24 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 01:01:25.056756 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83c1778-a99d-448d-b7fc-980a7dc45093 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:25.127691 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:25.161462 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83c1778-a99d-448d-b7fc-980a7dc45093 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:25.195825 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e93c96-20dd-405f-9b2b-dc3ec3cec30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:25.196475 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e93c96-20dd-405f-9b2b-dc3ec3cec30d None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:25.196750 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e93c96-20dd-405f-9b2b-dc3ec3cec30d None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:25.197017 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e93c96-20dd-405f-9b2b-dc3ec3cec30d None admin] PATH_INFO: `/v3/services/image` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:25.197621 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e93c96-20dd-405f-9b2b-dc3ec3cec30d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:25.199084 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e93c96-20dd-405f-9b2b-dc3ec3cec30d None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:25.203378 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1236d0-c771-4387-8a65-9374672a31d4 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:25.207882 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-55e93c96-20dd-405f-9b2b-dc3ec3cec30d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 01:01:25.207882 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:25.207882 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 01:01:25.207882 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 01:01:25.207882 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 01:01:25.207882 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 01:01:25.207882 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 01:01:25.207882 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 01:01:25.207882 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 01:01:25.207882 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:25.207882 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 01:01:25.207882 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:25.207882 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:25.207882 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:25.207882 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:25.207882 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:25.207882 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:25.209108 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:25.209108 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:25.209108 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:25.209108 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:25.209108 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:25.209108 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:25.209108 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 01:01:25.209108 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 01:01:25.209108 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 01:01:25.209108 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 01:01:25.209108 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:25.209108 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:25.209108 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:25.209108 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:25.209108 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:25.209108 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:25.210023 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:25.210023 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:25.210023 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:25.210023 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:25.210023 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:25.210023 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:25.210023 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:25.210023 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:25.210023 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:25.210023 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:25.210023 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 01:01:25.210023 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 01:01:25.210023 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 01:01:25.210023 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:25.214217 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 487/996] 10.222.0.42 () {56 vars in 1105 bytes} [Thu Sep 8 01:01:25 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:01:25.215115 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e93c96-20dd-405f-9b2b-dc3ec3cec30d None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:25.301800 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1236d0-c771-4387-8a65-9374672a31d4 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:25.313880 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e93c96-20dd-405f-9b2b-dc3ec3cec30d None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:25.335645 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e637f83-2d38-473b-8835-eeca952ad973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:25.336295 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e637f83-2d38-473b-8835-eeca952ad973 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:25.336568 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e637f83-2d38-473b-8835-eeca952ad973 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:25.336836 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e637f83-2d38-473b-8835-eeca952ad973 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:25.337412 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e637f83-2d38-473b-8835-eeca952ad973 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:25.338962 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e637f83-2d38-473b-8835-eeca952ad973 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:25.346388 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9060030f-f44f-4acb-bbe6-22df0862de15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:25.347005 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060030f-f44f-4acb-bbe6-22df0862de15 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:25.347283 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060030f-f44f-4acb-bbe6-22df0862de15 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:25.347452 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-2e637f83-2d38-473b-8835-eeca952ad973 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 01:01:25.347452 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:25.347452 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 01:01:25.347452 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 01:01:25.347452 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 01:01:25.347452 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 01:01:25.347452 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 01:01:25.347452 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 01:01:25.347452 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 01:01:25.347452 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:25.347452 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 01:01:25.347452 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:25.347452 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:25.347452 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:25.347452 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:25.347452 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:25.347452 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:25.352228 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:25.352228 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:25.352228 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:25.352228 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:25.352228 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:25.352228 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:25.352228 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 01:01:25.352228 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 01:01:25.352228 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 01:01:25.352228 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 01:01:25.352228 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:25.352228 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:25.352228 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:25.352228 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:25.352228 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:25.352228 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:25.353271 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:25.353271 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:25.353271 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:25.353271 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:25.353271 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:25.353271 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:25.353271 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:25.353271 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:25.353271 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:25.353271 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:25.353271 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 01:01:25.353271 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 01:01:25.353271 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 01:01:25.353271 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:25.354573 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060030f-f44f-4acb-bbe6-22df0862de15 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:25.354573 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9060030f-f44f-4acb-bbe6-22df0862de15 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:25.354720 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 510/997] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Sep 8 01:01:25 2022] GET /identity/v3/services/object-store => generated 93 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 01:01:25.354720 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e637f83-2d38-473b-8835-eeca952ad973 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:25.354864 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9060030f-f44f-4acb-bbe6-22df0862de15 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:25.357583 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:25.364985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-9060030f-f44f-4acb-bbe6-22df0862de15 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 01:01:25.364985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:25.364985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 01:01:25.364985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 01:01:25.364985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 01:01:25.364985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 01:01:25.364985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 01:01:25.364985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 01:01:25.364985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 01:01:25.364985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:25.364985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 01:01:25.364985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:25.364985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:25.364985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:25.364985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:25.364985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:25.364985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:25.366143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:25.366143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:25.366143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:25.366143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:25.366143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:25.366143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:25.366143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 01:01:25.366143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 01:01:25.366143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 01:01:25.366143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 01:01:25.366143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:25.366143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:25.366143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:25.366143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:25.366143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:25.366143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:25.366988 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:25.366988 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:25.366988 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:25.366988 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:25.366988 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:25.366988 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:25.366988 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:25.366988 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:25.366988 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:25.366988 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:25.366988 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 01:01:25.366988 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 01:01:25.366988 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 01:01:25.366988 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:25.366988 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 488/998] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 01:01:25 2022] GET /identity/v3/services/compute => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:01:25.371938 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9060030f-f44f-4acb-bbe6-22df0862de15 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:25.396831 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:25.456206 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e637f83-2d38-473b-8835-eeca952ad973 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:25.471913 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9060030f-f44f-4acb-bbe6-22df0862de15 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:25.489286 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cc6e1e-7a5a-47c7-82a0-a13d7f26a06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:25.489893 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc6e1e-7a5a-47c7-82a0-a13d7f26a06e None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:25.490176 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc6e1e-7a5a-47c7-82a0-a13d7f26a06e None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:25.490451 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc6e1e-7a5a-47c7-82a0-a13d7f26a06e None admin] PATH_INFO: `/v3/services` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:25.491078 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cc6e1e-7a5a-47c7-82a0-a13d7f26a06e None admin] RBAC: Adding query filter params (name=image) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:25.491380 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cc6e1e-7a5a-47c7-82a0-a13d7f26a06e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:25.492788 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cc6e1e-7a5a-47c7-82a0-a13d7f26a06e None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:25.497570 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 511/999] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:01:25 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:25.498334 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cc6e1e-7a5a-47c7-82a0-a13d7f26a06e None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:25.509162 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bb1225-9666-43e6-b463-89cb581ee6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:25.509789 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb1225-9666-43e6-b463-89cb581ee6ea None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:25.510595 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb1225-9666-43e6-b463-89cb581ee6ea None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:25.510871 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb1225-9666-43e6-b463-89cb581ee6ea None admin] PATH_INFO: `/v3/services` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:25.511753 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bb1225-9666-43e6-b463-89cb581ee6ea None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:25.512054 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bb1225-9666-43e6-b463-89cb581ee6ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:25.513446 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bb1225-9666-43e6-b463-89cb581ee6ea None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:25.518127 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 489/1000] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Sep 8 01:01:25 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:25.518885 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bb1225-9666-43e6-b463-89cb581ee6ea None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:25.580770 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:25.598893 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cc6e1e-7a5a-47c7-82a0-a13d7f26a06e None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:25.601231 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:25.622701 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bb1225-9666-43e6-b463-89cb581ee6ea None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:25.635730 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a55d9a-38ca-4e74-b880-9ca281694b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:25.636647 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a55d9a-38ca-4e74-b880-9ca281694b06 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:25.636927 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a55d9a-38ca-4e74-b880-9ca281694b06 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:25.637198 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a55d9a-38ca-4e74-b880-9ca281694b06 None admin] PATH_INFO: `/v3/services` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:25.637835 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a55d9a-38ca-4e74-b880-9ca281694b06 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:25.638137 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a55d9a-38ca-4e74-b880-9ca281694b06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:25.639510 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a55d9a-38ca-4e74-b880-9ca281694b06 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:25.644484 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 512/1001] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Sep 8 01:01:25 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:25.645256 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a55d9a-38ca-4e74-b880-9ca281694b06 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:25.657406 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c8669e-6bd0-4524-b203-7cd62b3b4086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:25.658007 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c8669e-6bd0-4524-b203-7cd62b3b4086 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:25.658288 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c8669e-6bd0-4524-b203-7cd62b3b4086 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:25.658554 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c8669e-6bd0-4524-b203-7cd62b3b4086 None admin] PATH_INFO: `/v3/services` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:25.659175 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c8669e-6bd0-4524-b203-7cd62b3b4086 None admin] RBAC: Adding query filter params (type=image) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:25.659474 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c8669e-6bd0-4524-b203-7cd62b3b4086 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:25.660899 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c8669e-6bd0-4524-b203-7cd62b3b4086 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:25.665159 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 490/1002] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:01:25 2022] GET /identity/v3/services?type=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:25.665952 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c8669e-6bd0-4524-b203-7cd62b3b4086 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:25.759025 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a55d9a-38ca-4e74-b880-9ca281694b06 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:25.790043 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c8669e-6bd0-4524-b203-7cd62b3b4086 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:25.801156 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96dffc4-6bbd-4cd8-b474-5ecf611ec28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:25.801774 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96dffc4-6bbd-4cd8-b474-5ecf611ec28f None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:25.802050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96dffc4-6bbd-4cd8-b474-5ecf611ec28f None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:25.802314 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96dffc4-6bbd-4cd8-b474-5ecf611ec28f None admin] PATH_INFO: `/v3/services` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:25.802951 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96dffc4-6bbd-4cd8-b474-5ecf611ec28f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:25.803254 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96dffc4-6bbd-4cd8-b474-5ecf611ec28f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:25.804679 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96dffc4-6bbd-4cd8-b474-5ecf611ec28f None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:25.811134 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 513/1003] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Sep 8 01:01:25 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:25.816866 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96dffc4-6bbd-4cd8-b474-5ecf611ec28f None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:25.822655 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:25.832067 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d59d60-3c90-493c-bf66-d615eacae106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:25.832708 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d59d60-3c90-493c-bf66-d615eacae106 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:25.832997 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d59d60-3c90-493c-bf66-d615eacae106 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:25.833270 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d59d60-3c90-493c-bf66-d615eacae106 None admin] PATH_INFO: `/v3/services` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:25.833906 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d59d60-3c90-493c-bf66-d615eacae106 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:25.834229 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d59d60-3c90-493c-bf66-d615eacae106 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:25.835604 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d59d60-3c90-493c-bf66-d615eacae106 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:25.841557 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 491/1004] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Sep 8 01:01:25 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:25.847492 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d59d60-3c90-493c-bf66-d615eacae106 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:25.858804 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:25.930571 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96dffc4-6bbd-4cd8-b474-5ecf611ec28f None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:25.961344 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d59d60-3c90-493c-bf66-d615eacae106 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:25.967875 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245eea47-0cb2-4d25-9c80-ac2598445d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:25.968518 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245eea47-0cb2-4d25-9c80-ac2598445d85 None admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:25.969059 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245eea47-0cb2-4d25-9c80-ac2598445d85 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:25.969059 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245eea47-0cb2-4d25-9c80-ac2598445d85 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:25.969706 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245eea47-0cb2-4d25-9c80-ac2598445d85 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:25.971347 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245eea47-0cb2-4d25-9c80-ac2598445d85 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:25.988757 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 514/1005] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:01:25 2022] POST /identity/v3/endpoints => generated 325 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 01:01:26.013766 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07997f4c-ab4a-4308-afbd-6b1068d0d0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:26.014315 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07997f4c-ab4a-4308-afbd-6b1068d0d0d3 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:26.014599 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07997f4c-ab4a-4308-afbd-6b1068d0d0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:26.014875 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07997f4c-ab4a-4308-afbd-6b1068d0d0d3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:26.015595 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07997f4c-ab4a-4308-afbd-6b1068d0d0d3 None admin] RBAC: Adding query filter params (interface=public, service_id=23b295db81ec4abba3e2ed8b2cd67494, region_id=RegionOne) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:26.015916 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07997f4c-ab4a-4308-afbd-6b1068d0d0d3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:26.017382 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07997f4c-ab4a-4308-afbd-6b1068d0d0d3 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:26.023628 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 492/1006] 10.222.0.42 () {56 vars in 1251 bytes} [Thu Sep 8 01:01:25 2022] GET /identity/v3/endpoints?service_id=23b295db81ec4abba3e2ed8b2cd67494&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:01:26.031567 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245eea47-0cb2-4d25-9c80-ac2598445d85 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:26.137549 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:26.150314 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245eea47-0cb2-4d25-9c80-ac2598445d85 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:26.191228 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023af76e-6fa9-4ec7-9246-8bce5310dcc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:26.191844 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023af76e-6fa9-4ec7-9246-8bce5310dcc1 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:26.192128 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023af76e-6fa9-4ec7-9246-8bce5310dcc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:26.192489 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023af76e-6fa9-4ec7-9246-8bce5310dcc1 None admin] PATH_INFO: `/v3/services` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:26.193101 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023af76e-6fa9-4ec7-9246-8bce5310dcc1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:26.194476 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023af76e-6fa9-4ec7-9246-8bce5310dcc1 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:26.201971 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 515/1007] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Sep 8 01:01:26 2022] GET /identity/v3/services => generated 1727 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:01:27.861017 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700dea2-06f5-4627-ae35-1deb1407dc81 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:27.861529 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700dea2-06f5-4627-ae35-1deb1407dc81 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:27.861607 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700dea2-06f5-4627-ae35-1deb1407dc81 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:27.863047 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 493/1008] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:01:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:01:27.868086 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797240e-a3b4-41b3-b73c-4e2edc2e55db None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:27.868517 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797240e-a3b4-41b3-b73c-4e2edc2e55db None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:27.868702 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797240e-a3b4-41b3-b73c-4e2edc2e55db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:27.888484 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-2797240e-a3b4-41b3-b73c-4e2edc2e55db None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:01:27.889604 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-2797240e-a3b4-41b3-b73c-4e2edc2e55db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:01:27.958135 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 516/1009] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:01:27 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 01:01:28.209546 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91021567-eb05-4c20-a63c-0d5688d1bc67 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:28.209841 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91021567-eb05-4c20-a63c-0d5688d1bc67 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:28.210018 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91021567-eb05-4c20-a63c-0d5688d1bc67 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:28.211485 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 494/1010] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:01:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:01:28.217523 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78587101-632b-442a-a34a-64ce1e85dd02 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:28.217809 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78587101-632b-442a-a34a-64ce1e85dd02 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:28.218078 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78587101-632b-442a-a34a-64ce1e85dd02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:28.227460 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91021567-eb05-4c20-a63c-0d5688d1bc67 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:28.236898 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-78587101-632b-442a-a34a-64ce1e85dd02 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:01:28.238123 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-78587101-632b-442a-a34a-64ce1e85dd02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:01:28.288081 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:28.315729 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 517/1011] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:01:28 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 01:01:28.340331 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91021567-eb05-4c20-a63c-0d5688d1bc67 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:28.366483 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62035404-5629-410a-a5ec-9b3dca5226fd None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:28.367217 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62035404-5629-410a-a5ec-9b3dca5226fd None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:28.367839 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62035404-5629-410a-a5ec-9b3dca5226fd None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:28.369355 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 518/1012] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:01:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 01:01:28.375321 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb42ba7-926c-457a-a264-33082a18667f None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:28.375462 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563deb2-e60c-4028-a229-b0ea9e1234ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:28.375958 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb42ba7-926c-457a-a264-33082a18667f None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:28.376083 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563deb2-e60c-4028-a229-b0ea9e1234ae None admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:28.376328 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563deb2-e60c-4028-a229-b0ea9e1234ae None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:28.376605 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563deb2-e60c-4028-a229-b0ea9e1234ae None admin] PATH_INFO: `/v3/services` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:28.376821 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb42ba7-926c-457a-a264-33082a18667f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:28.377266 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0563deb2-e60c-4028-a229-b0ea9e1234ae None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:28.378644 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0563deb2-e60c-4028-a229-b0ea9e1234ae None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:28.394884 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 495/1013] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Sep 8 01:01:28 2022] POST /identity/v3/services => generated 242 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 01:01:28.399651 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-5cb42ba7-926c-457a-a264-33082a18667f None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:01:28.401727 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-5cb42ba7-926c-457a-a264-33082a18667f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:01:28.416009 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:28.488184 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 519/1014] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:01:28 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 01:01:28.589754 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563deb2-e60c-4028-a229-b0ea9e1234ae None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:28.697824 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563deb2-e60c-4028-a229-b0ea9e1234ae None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:28.711635 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:28.735254 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ae2a0-7b78-42ff-b241-5da6abdc735b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:28.735943 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ae2a0-7b78-42ff-b241-5da6abdc735b None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:28.736253 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ae2a0-7b78-42ff-b241-5da6abdc735b None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:28.736524 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ae2a0-7b78-42ff-b241-5da6abdc735b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:28.737612 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1ae2a0-7b78-42ff-b241-5da6abdc735b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:28.739101 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1ae2a0-7b78-42ff-b241-5da6abdc735b None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:28.740813 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 496/1015] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:01:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:01:28.747170 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb42ba7-926c-457a-a264-33082a18667f None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:28.756638 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ae2a0-7b78-42ff-b241-5da6abdc735b None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:28.853003 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb42ba7-926c-457a-a264-33082a18667f None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:28.870207 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ae2a0-7b78-42ff-b241-5da6abdc735b None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:28.891517 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f15e50-f5cf-44b2-8fa9-f9ead6f15534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:28.892842 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f15e50-f5cf-44b2-8fa9-f9ead6f15534 None admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:28.893128 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f15e50-f5cf-44b2-8fa9-f9ead6f15534 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:28.893419 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f15e50-f5cf-44b2-8fa9-f9ead6f15534 None admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:28.894177 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f15e50-f5cf-44b2-8fa9-f9ead6f15534 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:28.895627 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f15e50-f5cf-44b2-8fa9-f9ead6f15534 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:28.910351 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28b45d1-3f97-4504-a7e4-64edafcb24c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:28.910995 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b45d1-3f97-4504-a7e4-64edafcb24c9 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:28.911304 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b45d1-3f97-4504-a7e4-64edafcb24c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:28.911566 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b45d1-3f97-4504-a7e4-64edafcb24c9 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:28.912183 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28b45d1-3f97-4504-a7e4-64edafcb24c9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:28.913561 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28b45d1-3f97-4504-a7e4-64edafcb24c9 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:28.914159 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:28.923744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-d28b45d1-3f97-4504-a7e4-64edafcb24c9 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 01:01:28.923744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:28.923744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 01:01:28.923744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 01:01:28.923744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 01:01:28.923744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 01:01:28.923744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 01:01:28.923744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 01:01:28.923744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 01:01:28.923744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:28.923744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 01:01:28.923744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:28.923744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:28.923744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:28.923744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:28.923744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:28.923744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:28.924866 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:28.924866 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:28.924866 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:28.924866 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:28.924866 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:28.924866 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:28.924866 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 01:01:28.924866 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 01:01:28.924866 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 01:01:28.924866 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 01:01:28.924866 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:28.924866 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:28.924866 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:28.924866 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:28.924866 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:28.924866 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:28.926400 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:28.926400 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:28.926400 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:28.926400 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:28.926400 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:28.926400 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:28.926400 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:28.926400 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:28.926400 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:28.926400 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:28.926400 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 01:01:28.926400 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 01:01:28.926400 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 01:01:28.926400 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:28.926400 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 497/1016] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 01:01:28 2022] GET /identity/v3/services/compute => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 01:01:28.940609 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 520/1017] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Sep 8 01:01:28 2022] POST /identity/v3/projects => generated 311 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:01:28.940838 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28b45d1-3f97-4504-a7e4-64edafcb24c9 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:28.992852 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:29.053676 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28b45d1-3f97-4504-a7e4-64edafcb24c9 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:29.092439 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02626f12-e1d2-4453-8806-7445e24c4600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:29.093062 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02626f12-e1d2-4453-8806-7445e24c4600 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:29.093347 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02626f12-e1d2-4453-8806-7445e24c4600 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:29.093623 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02626f12-e1d2-4453-8806-7445e24c4600 None admin] PATH_INFO: `/v3/services` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:29.094271 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02626f12-e1d2-4453-8806-7445e24c4600 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:29.094576 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02626f12-e1d2-4453-8806-7445e24c4600 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:29.095968 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02626f12-e1d2-4453-8806-7445e24c4600 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:29.102251 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 498/1018] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Sep 8 01:01:28 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:01:29.107288 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f15e50-f5cf-44b2-8fa9-f9ead6f15534 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:29.216629 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f15e50-f5cf-44b2-8fa9-f9ead6f15534 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:29.262226 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be3a31-1835-4481-8c16-bb9a058cb641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:29.262849 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be3a31-1835-4481-8c16-bb9a058cb641 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:29.263125 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be3a31-1835-4481-8c16-bb9a058cb641 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:29.263396 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be3a31-1835-4481-8c16-bb9a058cb641 None admin] PATH_INFO: `/v3/services` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:29.264030 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5be3a31-1835-4481-8c16-bb9a058cb641 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:29.264355 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5be3a31-1835-4481-8c16-bb9a058cb641 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:29.265729 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5be3a31-1835-4481-8c16-bb9a058cb641 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:29.279142 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 521/1019] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Sep 8 01:01:29 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:29.293777 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02626f12-e1d2-4453-8806-7445e24c4600 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:29.405720 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02626f12-e1d2-4453-8806-7445e24c4600 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:29.415480 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:29.444412 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd53b9c3-46b4-4d6b-8cf2-fc7a52f798e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:29.445395 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd53b9c3-46b4-4d6b-8cf2-fc7a52f798e9 None admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:29.446066 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd53b9c3-46b4-4d6b-8cf2-fc7a52f798e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:29.446540 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd53b9c3-46b4-4d6b-8cf2-fc7a52f798e9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:29.447420 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd53b9c3-46b4-4d6b-8cf2-fc7a52f798e9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:29.449055 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd53b9c3-46b4-4d6b-8cf2-fc7a52f798e9 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:29.473476 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 499/1020] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:01:29 2022] POST /identity/v3/endpoints => generated 334 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:01:30.605843 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd6329c-d19e-4bda-9346-6e9158ea0370 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:30.606395 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd6329c-d19e-4bda-9346-6e9158ea0370 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:30.606395 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd6329c-d19e-4bda-9346-6e9158ea0370 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:30.607698 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 522/1021] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:01:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:01:30.612819 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec6ca3-0480-4f90-b177-6811096f28f8 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:30.613204 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec6ca3-0480-4f90-b177-6811096f28f8 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:30.613371 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec6ca3-0480-4f90-b177-6811096f28f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:30.632267 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-53ec6ca3-0480-4f90-b177-6811096f28f8 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:01:30.633469 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-53ec6ca3-0480-4f90-b177-6811096f28f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:01:30.709435 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 500/1022] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:01:30 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 01:01:30.980014 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd6329c-d19e-4bda-9346-6e9158ea0370 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:31.041200 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:31.091113 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd6329c-d19e-4bda-9346-6e9158ea0370 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:31.125891 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0b808-9ff7-406f-b3cd-7151ce206f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:31.126523 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0b808-9ff7-406f-b3cd-7151ce206f26 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:31.126801 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0b808-9ff7-406f-b3cd-7151ce206f26 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:31.127072 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0b808-9ff7-406f-b3cd-7151ce206f26 None admin] PATH_INFO: `/v3/services/image` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:31.127652 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e0b808-9ff7-406f-b3cd-7151ce206f26 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:31.129043 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e0b808-9ff7-406f-b3cd-7151ce206f26 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:31.138681 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-c4e0b808-9ff7-406f-b3cd-7151ce206f26 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 01:01:31.138681 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:31.138681 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 01:01:31.138681 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 01:01:31.138681 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 01:01:31.138681 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 01:01:31.138681 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 01:01:31.138681 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 01:01:31.138681 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 01:01:31.138681 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:31.138681 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 01:01:31.138681 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:31.138681 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:31.138681 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:31.138681 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:31.138681 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:31.138681 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:31.139672 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:31.139672 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:31.139672 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:31.139672 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:31.139672 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:31.139672 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:31.139672 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 01:01:31.139672 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 01:01:31.139672 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 01:01:31.139672 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 01:01:31.139672 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:31.139672 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:31.139672 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:31.139672 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:31.139672 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:31.139672 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:31.140568 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:31.140568 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:31.140568 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:31.140568 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:31.140568 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:31.140568 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:31.140568 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:31.140568 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:31.140568 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:31.140568 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:31.140568 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 01:01:31.140568 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 01:01:31.140568 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 01:01:31.140568 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:31.144510 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 523/1023] 10.222.0.42 () {56 vars in 1105 bytes} [Thu Sep 8 01:01:30 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:01:31.145772 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ec6ca3-0480-4f90-b177-6811096f28f8 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:31.188944 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28750e-fecc-4fbf-aadf-946e324597d0 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:31.189233 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28750e-fecc-4fbf-aadf-946e324597d0 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:31.189507 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28750e-fecc-4fbf-aadf-946e324597d0 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:31.190834 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 524/1024] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:01:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 01:01:31.195862 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc87e957-e1b3-4421-9202-e3b16b07790c None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:31.196163 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc87e957-e1b3-4421-9202-e3b16b07790c None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:31.196448 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc87e957-e1b3-4421-9202-e3b16b07790c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:31.215366 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-cc87e957-e1b3-4421-9202-e3b16b07790c None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:01:31.216598 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-cc87e957-e1b3-4421-9202-e3b16b07790c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:01:31.237229 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:31.249594 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ec6ca3-0480-4f90-b177-6811096f28f8 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:31.284284 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac4e44-90b2-4dd9-bc32-9a796b2d5767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:31.284891 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac4e44-90b2-4dd9-bc32-9a796b2d5767 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:31.285170 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac4e44-90b2-4dd9-bc32-9a796b2d5767 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:31.285438 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac4e44-90b2-4dd9-bc32-9a796b2d5767 None admin] PATH_INFO: `/v3/services` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:31.286063 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cac4e44-90b2-4dd9-bc32-9a796b2d5767 None admin] RBAC: Adding query filter params (name=image) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:31.286369 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cac4e44-90b2-4dd9-bc32-9a796b2d5767 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:31.287739 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cac4e44-90b2-4dd9-bc32-9a796b2d5767 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:31.288788 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 525/1025] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:01:31 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 01:01:31.300996 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 501/1026] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:01:31 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:31.301964 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc87e957-e1b3-4421-9202-e3b16b07790c None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:31.354974 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:31.404001 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc87e957-e1b3-4421-9202-e3b16b07790c None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:31.437359 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e26c4-be50-4c08-bd28-b719fb263e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:31.437964 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e26c4-be50-4c08-bd28-b719fb263e30 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:31.438242 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e26c4-be50-4c08-bd28-b719fb263e30 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:31.438514 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e26c4-be50-4c08-bd28-b719fb263e30 None admin] PATH_INFO: `/v3/services` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:31.439141 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9e26c4-be50-4c08-bd28-b719fb263e30 None admin] RBAC: Adding query filter params (type=image) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:31.439446 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9e26c4-be50-4c08-bd28-b719fb263e30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:31.440841 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9e26c4-be50-4c08-bd28-b719fb263e30 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:31.451320 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 526/1027] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:01:31 2022] GET /identity/v3/services?type=image => generated 350 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:01:31.452128 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac4e44-90b2-4dd9-bc32-9a796b2d5767 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:31.559241 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e26c4-be50-4c08-bd28-b719fb263e30 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:31.568519 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac4e44-90b2-4dd9-bc32-9a796b2d5767 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:31.603907 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38cb168-eb03-448d-b1d1-f6a92f685905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:31.604716 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38cb168-eb03-448d-b1d1-f6a92f685905 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:31.604998 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38cb168-eb03-448d-b1d1-f6a92f685905 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:31.605271 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38cb168-eb03-448d-b1d1-f6a92f685905 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:31.605972 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38cb168-eb03-448d-b1d1-f6a92f685905 None admin] RBAC: Adding query filter params (interface=public, service_id=695712723a30404caef141ee5690862c, region_id=RegionOne) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:31.606279 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38cb168-eb03-448d-b1d1-f6a92f685905 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:31.607664 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38cb168-eb03-448d-b1d1-f6a92f685905 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:31.612827 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 502/1028] 10.222.0.42 () {56 vars in 1251 bytes} [Thu Sep 8 01:01:31 2022] GET /identity/v3/endpoints?service_id=695712723a30404caef141ee5690862c&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:01:31.621223 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38cb168-eb03-448d-b1d1-f6a92f685905 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:31.646499 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:31.678531 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e26c4-be50-4c08-bd28-b719fb263e30 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:31.700493 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:31.721601 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dd1316-385e-4dc3-9f6f-e64552541cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:31.722208 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd1316-385e-4dc3-9f6f-e64552541cdc None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:31.722484 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd1316-385e-4dc3-9f6f-e64552541cdc None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:31.722769 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd1316-385e-4dc3-9f6f-e64552541cdc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:31.723854 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dd1316-385e-4dc3-9f6f-e64552541cdc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:31.725317 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dd1316-385e-4dc3-9f6f-e64552541cdc None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:31.726898 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 527/1029] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:01:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:01:31.732687 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dd1316-385e-4dc3-9f6f-e64552541cdc None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:31.737823 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38cb168-eb03-448d-b1d1-f6a92f685905 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:31.773051 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d394944-3f69-4444-8e8f-f8a7276b3ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:31.773659 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d394944-3f69-4444-8e8f-f8a7276b3ac6 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:31.773936 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d394944-3f69-4444-8e8f-f8a7276b3ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:31.774206 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d394944-3f69-4444-8e8f-f8a7276b3ac6 None admin] PATH_INFO: `/v3/services` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:31.774805 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d394944-3f69-4444-8e8f-f8a7276b3ac6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:31.776846 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d394944-3f69-4444-8e8f-f8a7276b3ac6 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:31.783383 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 503/1030] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Sep 8 01:01:31 2022] GET /identity/v3/services => generated 1957 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:01:31.784759 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f34d30-7d8e-406f-8a0f-d2b02e9a36de None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:31.785043 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f34d30-7d8e-406f-8a0f-d2b02e9a36de None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:31.785319 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f34d30-7d8e-406f-8a0f-d2b02e9a36de None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:31.786536 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 504/1031] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:01:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:01:31.793362 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7085e8-0268-403f-9e20-d3ced4c04bf8 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:31.793641 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7085e8-0268-403f-9e20-d3ced4c04bf8 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:31.793917 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7085e8-0268-403f-9e20-d3ced4c04bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:31.812830 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-cf7085e8-0268-403f-9e20-d3ced4c04bf8 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:01:31.813994 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-cf7085e8-0268-403f-9e20-d3ced4c04bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:01:31.851516 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dd1316-385e-4dc3-9f6f-e64552541cdc None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:31.898349 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5f630c-16f0-4a1f-85b8-363fe2e547bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:31.899702 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f630c-16f0-4a1f-85b8-363fe2e547bb None admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:31.900184 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f630c-16f0-4a1f-85b8-363fe2e547bb None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:31.901137 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f630c-16f0-4a1f-85b8-363fe2e547bb None admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:31.902207 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5f630c-16f0-4a1f-85b8-363fe2e547bb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:31.904779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5f630c-16f0-4a1f-85b8-363fe2e547bb None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:31.914146 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 505/1032] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:01:31 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 01:01:31.932185 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 528/1033] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 8 01:01:31 2022] POST /identity/v3/users => generated 285 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 01:01:32.179871 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7085e8-0268-403f-9e20-d3ced4c04bf8 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:32.194944 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:32.282634 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7085e8-0268-403f-9e20-d3ced4c04bf8 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:32.317242 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6a42a-a629-4e25-bc93-791b4387ca45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:32.317883 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6a42a-a629-4e25-bc93-791b4387ca45 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:32.318164 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6a42a-a629-4e25-bc93-791b4387ca45 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:32.318436 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6a42a-a629-4e25-bc93-791b4387ca45 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:32.329451 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de6a42a-a629-4e25-bc93-791b4387ca45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:01:32.334312 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-5de6a42a-a629-4e25-bc93-791b4387ca45 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 01:01:32.334312 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:32.334312 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:32.334312 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:32.334312 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:32.334312 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:32.334312 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:32.334312 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:32.334312 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:32.334312 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:32.334312 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:32.334312 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:32.334312 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:01:32.334312 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:01:32.334312 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:01:32.334312 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:01:32.335238 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:01:32.335238 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:01:32.335238 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:01:32.335238 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:01:32.335238 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:32.335238 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:32.335238 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:32.335238 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:32.335238 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:32.335238 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:32.335238 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:32.335238 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:32.335238 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:32.335238 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:32.335238 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:32.335238 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:32.336992 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:32.336992 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:32.336992 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:32.336992 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:32.336992 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:01:32.336992 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:01:32.336992 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:01:32.336992 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:01:32.336992 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:01:32.336992 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:01:32.336992 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 01:01:32.336992 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:32.336992 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 506/1034] 10.222.0.42 () {56 vars in 1120 bytes} [Thu Sep 8 01:01:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:01:32.342304 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5f630c-16f0-4a1f-85b8-363fe2e547bb None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:32.385814 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:32.444172 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5f630c-16f0-4a1f-85b8-363fe2e547bb None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:32.478356 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cc456c-a46d-4558-9d52-c94d9cf3d079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:32.479113 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc456c-a46d-4558-9d52-c94d9cf3d079 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:32.479398 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc456c-a46d-4558-9d52-c94d9cf3d079 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:32.479775 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc456c-a46d-4558-9d52-c94d9cf3d079 None admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:32.480521 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cc456c-a46d-4558-9d52-c94d9cf3d079 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:32.480830 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cc456c-a46d-4558-9d52-c94d9cf3d079 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:32.482232 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cc456c-a46d-4558-9d52-c94d9cf3d079 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:32.492757 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 529/1035] 10.222.0.42 () {56 vars in 1115 bytes} [Thu Sep 8 01:01:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:32.497177 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6a42a-a629-4e25-bc93-791b4387ca45 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:32.570100 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:32.602567 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6a42a-a629-4e25-bc93-791b4387ca45 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:32.640174 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51030e27-a2b3-4d76-86ef-b51bf08ab31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:32.640817 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51030e27-a2b3-4d76-86ef-b51bf08ab31c None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:32.641099 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51030e27-a2b3-4d76-86ef-b51bf08ab31c None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:32.641373 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51030e27-a2b3-4d76-86ef-b51bf08ab31c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:32.652515 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51030e27-a2b3-4d76-86ef-b51bf08ab31c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:32.653971 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51030e27-a2b3-4d76-86ef-b51bf08ab31c None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:32.669709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-51030e27-a2b3-4d76-86ef-b51bf08ab31c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:01:32.669709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:32.669709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:32.669709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:32.669709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:32.669709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:32.669709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:32.669709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:32.669709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:32.669709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:32.669709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:32.669709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:32.669709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:01:32.669709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:01:32.669709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:01:32.669709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:01:32.670711 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:32.670711 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:32.670711 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:32.670711 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:32.670711 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:32.670711 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:32.670711 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:32.670711 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:32.670711 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:32.670711 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:32.670711 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:32.670711 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:32.670711 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:32.670711 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:32.670711 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:32.670711 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:32.670711 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:01:32.671648 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:01:32.671648 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:01:32.671648 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:32.672393 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 507/1036] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:01:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 01:01:32.676691 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cc456c-a46d-4558-9d52-c94d9cf3d079 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:32.782943 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cc456c-a46d-4558-9d52-c94d9cf3d079 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:32.784536 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:32.818172 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e018c5-b1e7-40a6-b7eb-ef43557c0f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:32.818775 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e018c5-b1e7-40a6-b7eb-ef43557c0f87 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:32.819061 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e018c5-b1e7-40a6-b7eb-ef43557c0f87 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:32.819333 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e018c5-b1e7-40a6-b7eb-ef43557c0f87 None admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:32.819978 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e018c5-b1e7-40a6-b7eb-ef43557c0f87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:32.820324 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e018c5-b1e7-40a6-b7eb-ef43557c0f87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:32.821691 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e018c5-b1e7-40a6-b7eb-ef43557c0f87 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:32.839173 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 530/1037] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:01:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:01:32.840355 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51030e27-a2b3-4d76-86ef-b51bf08ab31c None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:32.944049 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51030e27-a2b3-4d76-86ef-b51bf08ab31c None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:32.968284 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:32.980124 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55231e89-7d3c-4b97-ae94-a2d51603de66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:32.980778 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55231e89-7d3c-4b97-ae94-a2d51603de66 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:32.981060 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55231e89-7d3c-4b97-ae94-a2d51603de66 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:32.981331 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55231e89-7d3c-4b97-ae94-a2d51603de66 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:32.988983 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55231e89-7d3c-4b97-ae94-a2d51603de66 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:32.990420 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55231e89-7d3c-4b97-ae94-a2d51603de66 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:33.001532 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-55231e89-7d3c-4b97-ae94-a2d51603de66 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 01:01:33.001532 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:33.001532 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:33.001532 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:33.001532 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:33.001532 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:33.001532 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:33.001532 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:33.001532 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:33.001532 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:33.001532 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:33.001532 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:33.001532 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 01:01:33.001532 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 01:01:33.001532 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 01:01:33.001532 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 01:01:33.002527 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:33.002527 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:33.002527 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 01:01:33.002527 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:01:33.002527 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 01:01:33.002527 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:01:33.002527 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:33.002527 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:33.002527 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:33.002527 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:33.002527 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:33.002527 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:33.002527 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:33.002527 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:33.002527 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:33.002527 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:33.002527 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:33.002527 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:33.003504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:33.003504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:33.003504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 01:01:33.003504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 01:01:33.003504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 01:01:33.003504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 01:01:33.003504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 01:01:33.003504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 01:01:33.003504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 01:01:33.003504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:33.003504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 508/1038] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Sep 8 01:01:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:01:33.010961 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e018c5-b1e7-40a6-b7eb-ef43557c0f87 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:33.112361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e018c5-b1e7-40a6-b7eb-ef43557c0f87 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:33.147373 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713e7bfd-488d-4504-934c-34a5542930a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:33.148116 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e7bfd-488d-4504-934c-34a5542930a9 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:33.148554 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e7bfd-488d-4504-934c-34a5542930a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:33.148951 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e7bfd-488d-4504-934c-34a5542930a9 None admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:33.149717 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713e7bfd-488d-4504-934c-34a5542930a9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:33.150024 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713e7bfd-488d-4504-934c-34a5542930a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:33.151491 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713e7bfd-488d-4504-934c-34a5542930a9 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:33.168770 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 531/1039] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Sep 8 01:01:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:33.174556 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55231e89-7d3c-4b97-ae94-a2d51603de66 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:33.276663 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55231e89-7d3c-4b97-ae94-a2d51603de66 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:33.310652 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a97637-df48-4ec0-9f1b-28f93349a22e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:33.311255 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a97637-df48-4ec0-9f1b-28f93349a22e None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:33.311536 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a97637-df48-4ec0-9f1b-28f93349a22e None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:33.311809 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a97637-df48-4ec0-9f1b-28f93349a22e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:33.362713 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:33.380043 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a97637-df48-4ec0-9f1b-28f93349a22e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:33.381547 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a97637-df48-4ec0-9f1b-28f93349a22e None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:33.479500 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 509/1040] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 01:01:33 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 01:01:33.485227 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713e7bfd-488d-4504-934c-34a5542930a9 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:33.508319 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:33.589624 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713e7bfd-488d-4504-934c-34a5542930a9 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:33.631706 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f3cdf-64ad-4e99-8967-25e29ba98aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:33.632303 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f3cdf-64ad-4e99-8967-25e29ba98aa5 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:33.632581 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f3cdf-64ad-4e99-8967-25e29ba98aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:33.632850 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f3cdf-64ad-4e99-8967-25e29ba98aa5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:33.644149 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6f3cdf-64ad-4e99-8967-25e29ba98aa5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:33.645704 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6f3cdf-64ad-4e99-8967-25e29ba98aa5 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:33.661361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-7b6f3cdf-64ad-4e99-8967-25e29ba98aa5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:01:33.661361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:33.661361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:33.661361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:33.661361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:33.661361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:33.661361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:33.661361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:33.661361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:33.661361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:33.661361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:33.661361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:33.661361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:01:33.661361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:01:33.661361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:01:33.661361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:01:33.662385 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:33.662385 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:33.662385 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:33.662385 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:33.662385 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:33.662385 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:33.662385 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:33.662385 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:33.662385 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:33.662385 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:33.662385 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:33.662385 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:33.662385 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:33.662385 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:33.662385 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:33.662385 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:33.662385 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:01:33.663321 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:01:33.663321 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:01:33.663321 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:33.663321 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 532/1041] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:01:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 01:01:33.670082 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a97637-df48-4ec0-9f1b-28f93349a22e None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:33.756403 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:33.774557 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a97637-df48-4ec0-9f1b-28f93349a22e None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:33.809124 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3328efa0-4f6f-463c-9b70-59681229244e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:33.809730 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328efa0-4f6f-463c-9b70-59681229244e None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:33.810010 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328efa0-4f6f-463c-9b70-59681229244e None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:33.810282 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328efa0-4f6f-463c-9b70-59681229244e None admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:33.810929 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3328efa0-4f6f-463c-9b70-59681229244e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:33.811230 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3328efa0-4f6f-463c-9b70-59681229244e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:33.812646 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3328efa0-4f6f-463c-9b70-59681229244e None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:33.829556 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 510/1042] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:01:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:33.831076 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f3cdf-64ad-4e99-8967-25e29ba98aa5 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:33.908118 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:33.941977 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f3cdf-64ad-4e99-8967-25e29ba98aa5 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:33.975559 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3ddbb9-5088-4c65-983e-0034eb8e1973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:33.976211 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3ddbb9-5088-4c65-983e-0034eb8e1973 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:33.976494 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3ddbb9-5088-4c65-983e-0034eb8e1973 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:33.976757 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3ddbb9-5088-4c65-983e-0034eb8e1973 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:33.984285 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3ddbb9-5088-4c65-983e-0034eb8e1973 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:33.985722 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3ddbb9-5088-4c65-983e-0034eb8e1973 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:33.997060 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-9e3ddbb9-5088-4c65-983e-0034eb8e1973 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:01:33.997060 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:33.997060 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:33.997060 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:33.997060 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:33.997060 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:33.997060 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:33.997060 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:33.997060 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:33.997060 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:33.997060 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:33.997060 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:33.997060 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 01:01:33.997060 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 01:01:33.997060 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 01:01:33.997959 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 01:01:33.997959 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:33.997959 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:33.997959 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:33.997959 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:33.997959 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:33.997959 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:33.997959 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:33.997959 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:33.997959 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:33.997959 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:33.997959 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:33.997959 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:33.997959 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:33.997959 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:33.997959 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:33.997959 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:33.998871 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 01:01:33.998871 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 01:01:33.998871 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 01:01:33.998871 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 01:01:33.998871 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 01:01:33.998871 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 01:01:33.998871 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:01:33.998871 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:33.999910 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 533/1043] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 01:01:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 01:01:34.003998 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3328efa0-4f6f-463c-9b70-59681229244e None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:34.005502 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26740d36-c5bf-459d-a96c-c51101722fd6 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:34.005786 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26740d36-c5bf-459d-a96c-c51101722fd6 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:34.006059 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26740d36-c5bf-459d-a96c-c51101722fd6 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:34.007353 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 534/1044] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:01:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:01:34.014083 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca27398-3b3b-4c1f-96be-7fe11e134ed0 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:34.014362 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca27398-3b3b-4c1f-96be-7fe11e134ed0 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:34.014637 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca27398-3b3b-4c1f-96be-7fe11e134ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:34.033124 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-aca27398-3b3b-4c1f-96be-7fe11e134ed0 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:01:34.034317 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-aca27398-3b3b-4c1f-96be-7fe11e134ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:01:34.110407 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 535/1045] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:01:34 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 01:01:34.125168 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3328efa0-4f6f-463c-9b70-59681229244e None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:34.161383 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d45eb0-9df4-4a7f-b2ef-5405046b5fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:34.161974 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d45eb0-9df4-4a7f-b2ef-5405046b5fb6 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:34.162257 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d45eb0-9df4-4a7f-b2ef-5405046b5fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:34.162532 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d45eb0-9df4-4a7f-b2ef-5405046b5fb6 None admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:34.163276 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d45eb0-9df4-4a7f-b2ef-5405046b5fb6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:34.163589 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d45eb0-9df4-4a7f-b2ef-5405046b5fb6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:34.165054 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d45eb0-9df4-4a7f-b2ef-5405046b5fb6 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:34.183767 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 511/1046] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 8 01:01:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:01:34.184916 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca27398-3b3b-4c1f-96be-7fe11e134ed0 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:34.195239 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedf7382-7990-437f-8085-497931dfd3df None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:34.195636 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedf7382-7990-437f-8085-497931dfd3df None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:34.195911 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedf7382-7990-437f-8085-497931dfd3df None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:34.197591 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 512/1047] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:01:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 01:01:34.202706 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f4994e-ffd0-45fe-975c-dd83523bd916 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:34.202987 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f4994e-ffd0-45fe-975c-dd83523bd916 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:34.203259 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f4994e-ffd0-45fe-975c-dd83523bd916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:34.211299 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:34.225415 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-f1f4994e-ffd0-45fe-975c-dd83523bd916 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:01:34.226602 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-f1f4994e-ffd0-45fe-975c-dd83523bd916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:01:34.250481 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:34.294250 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca27398-3b3b-4c1f-96be-7fe11e134ed0 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:34.301894 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 513/1048] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:01:34 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 01:01:34.329910 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57504b9f-0bd2-4a3b-9bc8-8bfdd808cd1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:34.330521 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57504b9f-0bd2-4a3b-9bc8-8bfdd808cd1e None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:34.330801 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57504b9f-0bd2-4a3b-9bc8-8bfdd808cd1e None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:34.331076 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57504b9f-0bd2-4a3b-9bc8-8bfdd808cd1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:34.331797 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57504b9f-0bd2-4a3b-9bc8-8bfdd808cd1e None admin] RBAC: Adding query filter params (role.id=62b04068ecbd4b13a405370cb199a142, scope.project.id=78ad2a8fb5264fb4af935e039d28c8de, user.id=ae643cfdcb164aa9b474747cd21aa116) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:34.332105 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57504b9f-0bd2-4a3b-9bc8-8bfdd808cd1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:34.333593 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57504b9f-0bd2-4a3b-9bc8-8bfdd808cd1e None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:34.344172 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 536/1049] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 8 01:01:34 2022] GET /identity/v3/role_assignments?user.id=ae643cfdcb164aa9b474747cd21aa116&scope.project.id=78ad2a8fb5264fb4af935e039d28c8de&role.id=62b04068ecbd4b13a405370cb199a142 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:01:34.383477 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f4994e-ffd0-45fe-975c-dd83523bd916 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:34.493347 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f4994e-ffd0-45fe-975c-dd83523bd916 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:34.532119 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a087586e-e1f8-4bb8-b313-6acd4cacd108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:34.532769 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a087586e-e1f8-4bb8-b313-6acd4cacd108 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:34.533045 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a087586e-e1f8-4bb8-b313-6acd4cacd108 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:34.533313 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a087586e-e1f8-4bb8-b313-6acd4cacd108 None admin] PATH_INFO: `/v3/services/image` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:34.533891 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a087586e-e1f8-4bb8-b313-6acd4cacd108 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:34.535251 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a087586e-e1f8-4bb8-b313-6acd4cacd108 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:34.537130 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:34.545631 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-a087586e-e1f8-4bb8-b313-6acd4cacd108 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 01:01:34.545631 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:34.545631 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 01:01:34.545631 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 01:01:34.545631 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 01:01:34.545631 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 01:01:34.545631 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 01:01:34.545631 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 01:01:34.545631 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 01:01:34.545631 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:34.545631 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 01:01:34.545631 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:34.545631 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:34.545631 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:34.545631 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:34.545631 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:34.545631 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:34.546639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:34.546639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:34.546639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:34.546639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:34.546639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:34.546639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:34.546639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 01:01:34.546639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 01:01:34.546639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 01:01:34.546639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 01:01:34.546639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:34.546639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:34.546639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:34.546639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:34.546639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:34.546639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:34.547518 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:34.547518 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:34.547518 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:34.547518 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:34.547518 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:34.547518 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:34.547518 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:34.547518 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:34.547518 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:34.547518 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:34.547518 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 01:01:34.547518 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 01:01:34.547518 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 01:01:34.547518 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:34.547518 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 514/1050] 10.222.0.42 () {56 vars in 1105 bytes} [Thu Sep 8 01:01:34 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:01:34.554054 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57504b9f-0bd2-4a3b-9bc8-8bfdd808cd1e None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:34.577102 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a087586e-e1f8-4bb8-b313-6acd4cacd108 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:34.665309 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57504b9f-0bd2-4a3b-9bc8-8bfdd808cd1e None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:34.693629 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:34.704500 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a087586e-e1f8-4bb8-b313-6acd4cacd108 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:34.705746 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9cc470-543d-4a1a-8010-8199434b408a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:34.706427 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9cc470-543d-4a1a-8010-8199434b408a None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:34.707456 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9cc470-543d-4a1a-8010-8199434b408a None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:34.707456 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9cc470-543d-4a1a-8010-8199434b408a None admin] PATH_INFO: `/v3/services` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:34.707749 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9cc470-543d-4a1a-8010-8199434b408a None admin] RBAC: Adding query filter params (name=image) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:34.707983 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9cc470-543d-4a1a-8010-8199434b408a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:34.709357 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9cc470-543d-4a1a-8010-8199434b408a None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:34.719447 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 537/1051] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:01:34 2022] GET /identity/v3/services?name=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:34.720096 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9cc470-543d-4a1a-8010-8199434b408a None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:34.741316 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cb28a1-c4bf-4f87-8742-12aea6f68539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:34.741949 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb28a1-c4bf-4f87-8742-12aea6f68539 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:34.742230 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb28a1-c4bf-4f87-8742-12aea6f68539 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:34.742501 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb28a1-c4bf-4f87-8742-12aea6f68539 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:34.753848 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cb28a1-c4bf-4f87-8742-12aea6f68539 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:01:34.758608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-a5cb28a1-c4bf-4f87-8742-12aea6f68539 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 01:01:34.758608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:34.758608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:34.758608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:34.758608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:34.758608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:34.758608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:34.758608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:34.758608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:34.758608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:34.758608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:34.758608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:34.758608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:01:34.758608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:01:34.758608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:01:34.758608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:01:34.759709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:01:34.759709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:01:34.759709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:01:34.759709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:01:34.759709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:34.759709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:34.759709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:34.759709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:34.759709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:34.759709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:34.759709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:34.759709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:34.759709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:34.759709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:34.759709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:34.759709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:34.761166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:34.761166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:34.761166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:34.761166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:34.761166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:01:34.761166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:01:34.761166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:01:34.761166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:01:34.761166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:01:34.761166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:01:34.761166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 01:01:34.761166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:34.761166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 515/1052] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 8 01:01:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 01:01:34.767237 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cb28a1-c4bf-4f87-8742-12aea6f68539 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:34.834422 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9cc470-543d-4a1a-8010-8199434b408a None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:34.838349 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:34.873696 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e0b4e1-5682-4b12-9e16-2295da26b90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:34.874303 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e0b4e1-5682-4b12-9e16-2295da26b90e None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:34.874581 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e0b4e1-5682-4b12-9e16-2295da26b90e None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:34.874849 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e0b4e1-5682-4b12-9e16-2295da26b90e None admin] PATH_INFO: `/v3/services` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:34.875478 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e0b4e1-5682-4b12-9e16-2295da26b90e None admin] RBAC: Adding query filter params (type=image) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:34.875780 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e0b4e1-5682-4b12-9e16-2295da26b90e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:34.877175 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e0b4e1-5682-4b12-9e16-2295da26b90e None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:34.886905 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 538/1053] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:01:34 2022] GET /identity/v3/services?type=image => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:34.887469 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cb28a1-c4bf-4f87-8742-12aea6f68539 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:34.887880 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e0b4e1-5682-4b12-9e16-2295da26b90e None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:34.925668 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43869fa-84c2-45e0-80f9-968ce1920f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:34.926273 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43869fa-84c2-45e0-80f9-968ce1920f4e None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:34.926552 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43869fa-84c2-45e0-80f9-968ce1920f4e None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:34.926820 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43869fa-84c2-45e0-80f9-968ce1920f4e None admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:34.927476 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43869fa-84c2-45e0-80f9-968ce1920f4e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:34.927779 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43869fa-84c2-45e0-80f9-968ce1920f4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:34.929323 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43869fa-84c2-45e0-80f9-968ce1920f4e None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:34.938682 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 516/1054] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 01:01:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:01:34.944399 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43869fa-84c2-45e0-80f9-968ce1920f4e None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:34.945810 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:35.000588 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e0b4e1-5682-4b12-9e16-2295da26b90e None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:35.039195 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710ae355-8fe6-4c1c-a2bb-ff298fe52779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:35.039718 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ae355-8fe6-4c1c-a2bb-ff298fe52779 None admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:35.039995 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ae355-8fe6-4c1c-a2bb-ff298fe52779 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:35.040291 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ae355-8fe6-4c1c-a2bb-ff298fe52779 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:35.040961 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710ae355-8fe6-4c1c-a2bb-ff298fe52779 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:35.042576 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710ae355-8fe6-4c1c-a2bb-ff298fe52779 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:35.049790 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43869fa-84c2-45e0-80f9-968ce1920f4e None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:35.076165 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 539/1055] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:01:34 2022] POST /identity/v3/endpoints => generated 327 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:01:35.081630 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:35.107732 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265a4e1b-ade8-4af8-af8c-893f02f5004e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:35.108600 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a4e1b-ade8-4af8-af8c-893f02f5004e None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:35.108977 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a4e1b-ade8-4af8-af8c-893f02f5004e None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:35.109343 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a4e1b-ade8-4af8-af8c-893f02f5004e None admin] PATH_INFO: `/v3/users/f07010516657452bad5de86fc97ee8bc` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:35.127498 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265a4e1b-ade8-4af8-af8c-893f02f5004e None admin] RBAC: Authorizing `identity:get_user(user_id=f07010516657452bad5de86fc97ee8bc)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:35.129279 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265a4e1b-ade8-4af8-af8c-893f02f5004e None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:35.136772 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 517/1056] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 8 01:01:34 2022] GET /identity/v3/users/f07010516657452bad5de86fc97ee8bc => generated 285 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:01:35.138080 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710ae355-8fe6-4c1c-a2bb-ff298fe52779 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:35.252448 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710ae355-8fe6-4c1c-a2bb-ff298fe52779 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:35.265410 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:35.292612 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb6bc98-04fb-4627-b2b2-8ae86154aa4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:35.293147 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb6bc98-04fb-4627-b2b2-8ae86154aa4f None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:35.293422 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb6bc98-04fb-4627-b2b2-8ae86154aa4f None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:35.293688 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb6bc98-04fb-4627-b2b2-8ae86154aa4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:35.362806 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb6bc98-04fb-4627-b2b2-8ae86154aa4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:35.364285 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb6bc98-04fb-4627-b2b2-8ae86154aa4f None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:35.466496 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 540/1057] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 01:01:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 01:01:35.468508 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265a4e1b-ade8-4af8-af8c-893f02f5004e None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:35.571897 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265a4e1b-ade8-4af8-af8c-893f02f5004e None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:35.612245 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e2cfbd-b8fd-4029-af10-7040a357afdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:35.612875 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e2cfbd-b8fd-4029-af10-7040a357afdb None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:35.613154 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e2cfbd-b8fd-4029-af10-7040a357afdb None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:35.613426 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e2cfbd-b8fd-4029-af10-7040a357afdb None admin] PATH_INFO: `/v3/projects/b898a6e4e2da4c09893c6aa1db674bd4` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:35.614563 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e2cfbd-b8fd-4029-af10-7040a357afdb None admin] RBAC: Authorizing `identity:get_project(project_id=b898a6e4e2da4c09893c6aa1db674bd4)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:35.616014 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e2cfbd-b8fd-4029-af10-7040a357afdb None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:35.623500 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 518/1058] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 01:01:35 2022] GET /identity/v3/projects/b898a6e4e2da4c09893c6aa1db674bd4 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:35.625384 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb6bc98-04fb-4627-b2b2-8ae86154aa4f None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:35.637889 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:35.729291 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb6bc98-04fb-4627-b2b2-8ae86154aa4f None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:35.763642 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b5319-65d9-4ed6-a529-c40b5b89cdc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:35.764266 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b5319-65d9-4ed6-a529-c40b5b89cdc8 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:35.764547 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b5319-65d9-4ed6-a529-c40b5b89cdc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:35.764816 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b5319-65d9-4ed6-a529-c40b5b89cdc8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:35.765537 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42b5319-65d9-4ed6-a529-c40b5b89cdc8 None admin] RBAC: Adding query filter params (role.id=34de369ec7c54715aa83a9052a26c3f2, scope.project.id=b898a6e4e2da4c09893c6aa1db674bd4, user.id=f07010516657452bad5de86fc97ee8bc) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:35.765844 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42b5319-65d9-4ed6-a529-c40b5b89cdc8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:35.767266 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42b5319-65d9-4ed6-a529-c40b5b89cdc8 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:35.780414 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 541/1059] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 8 01:01:35 2022] GET /identity/v3/role_assignments?user.id=f07010516657452bad5de86fc97ee8bc&scope.project.id=b898a6e4e2da4c09893c6aa1db674bd4&role.id=34de369ec7c54715aa83a9052a26c3f2 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:01:36.500677 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6908d3b-440f-4bf1-9225-e9026e52bb09 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:36.501199 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6908d3b-440f-4bf1-9225-e9026e52bb09 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:36.501199 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6908d3b-440f-4bf1-9225-e9026e52bb09 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:36.502534 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 519/1060] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:01:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:01:36.507745 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680ad3a-fd3b-42dc-bf6d-d4b72a8e809c None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:36.508124 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680ad3a-fd3b-42dc-bf6d-d4b72a8e809c None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:36.508331 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680ad3a-fd3b-42dc-bf6d-d4b72a8e809c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:36.527207 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-2680ad3a-fd3b-42dc-bf6d-d4b72a8e809c None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:01:36.528411 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-2680ad3a-fd3b-42dc-bf6d-d4b72a8e809c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:01:36.595213 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 542/1061] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:01:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 01:01:36.862452 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6908d3b-440f-4bf1-9225-e9026e52bb09 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:36.908605 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:36.969728 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6908d3b-440f-4bf1-9225-e9026e52bb09 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:37.005765 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c9536-187c-4c6a-8f79-a31b18dc58f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:37.006396 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c9536-187c-4c6a-8f79-a31b18dc58f3 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:37.006670 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c9536-187c-4c6a-8f79-a31b18dc58f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:37.006936 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c9536-187c-4c6a-8f79-a31b18dc58f3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:37.017840 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40c9536-187c-4c6a-8f79-a31b18dc58f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:01:37.022568 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-e40c9536-187c-4c6a-8f79-a31b18dc58f3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 01:01:37.022568 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:37.022568 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:37.022568 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:37.022568 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:37.022568 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:37.022568 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:37.022568 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:37.022568 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:37.022568 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:37.022568 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:37.022568 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:37.022568 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:01:37.022568 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:01:37.022568 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:01:37.022568 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:01:37.023467 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:01:37.023467 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:01:37.023467 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:01:37.023467 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:01:37.023467 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:37.023467 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:37.023467 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:37.023467 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:37.023467 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:37.023467 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:37.023467 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:37.023467 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:37.023467 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:37.023467 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:37.023467 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:37.023467 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:37.028950 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:37.028950 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:37.028950 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:37.028950 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:37.028950 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:01:37.028950 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:01:37.028950 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:01:37.028950 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:01:37.028950 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:01:37.028950 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:01:37.028950 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 01:01:37.028950 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:37.028950 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 520/1062] 10.222.0.42 () {56 vars in 1120 bytes} [Thu Sep 8 01:01:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:01:37.030396 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680ad3a-fd3b-42dc-bf6d-d4b72a8e809c None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:37.034490 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:37.133948 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680ad3a-fd3b-42dc-bf6d-d4b72a8e809c None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:37.170198 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc35ee65-9b88-4d1e-a04f-c6fd523ec87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:37.170833 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc35ee65-9b88-4d1e-a04f-c6fd523ec87d None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:37.171112 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc35ee65-9b88-4d1e-a04f-c6fd523ec87d None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:37.171386 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc35ee65-9b88-4d1e-a04f-c6fd523ec87d None admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:37.172086 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc35ee65-9b88-4d1e-a04f-c6fd523ec87d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:37.172426 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc35ee65-9b88-4d1e-a04f-c6fd523ec87d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:37.173850 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc35ee65-9b88-4d1e-a04f-c6fd523ec87d None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:37.184208 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 543/1063] 10.222.0.42 () {56 vars in 1115 bytes} [Thu Sep 8 01:01:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:37.189885 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c9536-187c-4c6a-8f79-a31b18dc58f3 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:37.294130 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f949db-fff5-499d-adf8-625c26e435aa None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:37.294488 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f949db-fff5-499d-adf8-625c26e435aa None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:37.294688 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f949db-fff5-499d-adf8-625c26e435aa None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:37.294870 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c9536-187c-4c6a-8f79-a31b18dc58f3 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:37.296190 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 544/1064] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:01:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 01:01:37.296658 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:37.301287 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4d8b4-877a-4e41-9d6b-855436a0afc6 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:37.301576 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4d8b4-877a-4e41-9d6b-855436a0afc6 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:37.301852 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4d8b4-877a-4e41-9d6b-855436a0afc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:37.320271 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-18d4d8b4-877a-4e41-9d6b-855436a0afc6 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:01:37.321469 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-18d4d8b4-877a-4e41-9d6b-855436a0afc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:01:37.336407 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c4e820-dbed-4582-9a9b-123df085c54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:37.337316 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c4e820-dbed-4582-9a9b-123df085c54d None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:37.337601 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c4e820-dbed-4582-9a9b-123df085c54d None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:37.338064 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c4e820-dbed-4582-9a9b-123df085c54d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:37.349185 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c4e820-dbed-4582-9a9b-123df085c54d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:37.350649 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c4e820-dbed-4582-9a9b-123df085c54d None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:37.365999 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-16c4e820-dbed-4582-9a9b-123df085c54d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:01:37.365999 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:37.365999 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:37.365999 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:37.365999 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:37.365999 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:37.365999 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:37.365999 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:37.365999 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:37.365999 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:37.365999 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:37.365999 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:37.365999 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:01:37.365999 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:01:37.365999 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:01:37.365999 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:01:37.366980 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:37.366980 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:37.366980 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:37.366980 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:37.366980 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:37.366980 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:37.366980 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:37.366980 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:37.366980 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:37.366980 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:37.366980 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:37.366980 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:37.366980 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:37.366980 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:37.366980 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:37.366980 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:37.366980 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:01:37.373182 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:01:37.373182 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:01:37.373182 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:37.373182 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 521/1065] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:01:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:01:37.373427 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c4e820-dbed-4582-9a9b-123df085c54d None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:37.391051 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:37.400170 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 545/1066] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:01:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 01:01:37.482603 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c4e820-dbed-4582-9a9b-123df085c54d None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:37.516671 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b62c4b4-7e8e-4741-9277-503a66540a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:37.517319 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b62c4b4-7e8e-4741-9277-503a66540a98 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:37.517554 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b62c4b4-7e8e-4741-9277-503a66540a98 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:37.517804 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b62c4b4-7e8e-4741-9277-503a66540a98 None admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:37.518445 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b62c4b4-7e8e-4741-9277-503a66540a98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:37.518750 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b62c4b4-7e8e-4741-9277-503a66540a98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:37.520127 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b62c4b4-7e8e-4741-9277-503a66540a98 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:37.532564 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 522/1067] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:01:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:01:37.537995 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d4d8b4-877a-4e41-9d6b-855436a0afc6 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:37.642203 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d4d8b4-877a-4e41-9d6b-855436a0afc6 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:37.671609 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b62c4b4-7e8e-4741-9277-503a66540a98 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:37.679195 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e900d5cb-ca76-4969-9675-43858933ad5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:37.679824 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900d5cb-ca76-4969-9675-43858933ad5b None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:37.680100 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900d5cb-ca76-4969-9675-43858933ad5b None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:37.680397 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900d5cb-ca76-4969-9675-43858933ad5b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:37.688388 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:37.689364 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e900d5cb-ca76-4969-9675-43858933ad5b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:37.691026 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e900d5cb-ca76-4969-9675-43858933ad5b None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:37.703040 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-e900d5cb-ca76-4969-9675-43858933ad5b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 01:01:37.703040 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:37.703040 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:37.703040 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:37.703040 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:37.703040 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:37.703040 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:37.703040 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:37.703040 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:37.703040 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:37.703040 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:37.703040 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:37.703040 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 01:01:37.703040 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 01:01:37.703040 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 01:01:37.703040 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 01:01:37.704204 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:37.704204 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:37.704204 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 01:01:37.704204 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:01:37.704204 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 01:01:37.704204 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:01:37.704204 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:37.704204 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:37.704204 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:37.704204 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:37.704204 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:37.704204 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:37.704204 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:37.704204 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:37.704204 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:37.704204 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:37.704204 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:37.704204 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:37.705238 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:37.705238 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:37.705238 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 01:01:37.705238 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 01:01:37.705238 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 01:01:37.705238 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 01:01:37.705238 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 01:01:37.705238 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 01:01:37.705238 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 01:01:37.705238 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:37.705238 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 546/1068] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Sep 8 01:01:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:01:37.710057 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e900d5cb-ca76-4969-9675-43858933ad5b None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:37.771817 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:37.784201 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b62c4b4-7e8e-4741-9277-503a66540a98 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:37.823137 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e900d5cb-ca76-4969-9675-43858933ad5b None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:37.823517 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1524268-acc2-45c8-a950-171083b364be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:37.824158 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1524268-acc2-45c8-a950-171083b364be None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:37.824443 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1524268-acc2-45c8-a950-171083b364be None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:37.824721 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1524268-acc2-45c8-a950-171083b364be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:37.860770 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c274b08-4ee2-48b8-9e01-a2ac74a0f813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:37.862125 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c274b08-4ee2-48b8-9e01-a2ac74a0f813 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:37.862407 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c274b08-4ee2-48b8-9e01-a2ac74a0f813 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:37.862678 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c274b08-4ee2-48b8-9e01-a2ac74a0f813 None admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:37.863412 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c274b08-4ee2-48b8-9e01-a2ac74a0f813 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:37.863716 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c274b08-4ee2-48b8-9e01-a2ac74a0f813 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:37.865186 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c274b08-4ee2-48b8-9e01-a2ac74a0f813 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:37.889703 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 547/1069] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Sep 8 01:01:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:01:37.890938 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c274b08-4ee2-48b8-9e01-a2ac74a0f813 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:37.902383 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1524268-acc2-45c8-a950-171083b364be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:37.903778 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1524268-acc2-45c8-a950-171083b364be None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:37.919113 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:37.995235 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c274b08-4ee2-48b8-9e01-a2ac74a0f813 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:38.009150 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:38.011710 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 523/1070] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 01:01:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 08 01:01:38.017866 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed2f85-2258-452a-acc1-f289cecb9ec0 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:38.018149 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed2f85-2258-452a-acc1-f289cecb9ec0 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:38.018417 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed2f85-2258-452a-acc1-f289cecb9ec0 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:38.019677 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 524/1071] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:01:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:01:38.020375 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ed2f85-2258-452a-acc1-f289cecb9ec0 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:38.031455 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c28c7ab-34a2-48d3-9d0e-df8e1dadb731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:38.032081 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28c7ab-34a2-48d3-9d0e-df8e1dadb731 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:38.032399 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28c7ab-34a2-48d3-9d0e-df8e1dadb731 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:38.032671 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28c7ab-34a2-48d3-9d0e-df8e1dadb731 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:38.042663 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c28c7ab-34a2-48d3-9d0e-df8e1dadb731 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:38.044087 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c28c7ab-34a2-48d3-9d0e-df8e1dadb731 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:38.058380 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-6c28c7ab-34a2-48d3-9d0e-df8e1dadb731 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:01:38.058380 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:38.058380 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:38.058380 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:38.058380 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:38.058380 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:38.058380 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:38.058380 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:38.058380 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:38.058380 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:38.058380 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:38.058380 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:38.058380 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:01:38.058380 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:01:38.058380 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:01:38.058380 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:01:38.059481 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:38.059481 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:38.059481 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:38.059481 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:38.059481 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:38.059481 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:38.059481 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:38.059481 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:38.059481 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:38.059481 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:38.059481 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:38.059481 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:38.059481 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:38.059481 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:38.059481 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:38.059481 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:38.059481 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:01:38.060432 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:01:38.060432 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:01:38.060432 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:38.060432 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 548/1072] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:01:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:01:38.060927 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4d934b-41d9-40fc-90ff-0663a00f46c7 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:38.061102 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4d934b-41d9-40fc-90ff-0663a00f46c7 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:38.061369 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4d934b-41d9-40fc-90ff-0663a00f46c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:38.080271 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-8b4d934b-41d9-40fc-90ff-0663a00f46c7 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:01:38.081458 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-8b4d934b-41d9-40fc-90ff-0663a00f46c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:01:38.122390 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ed2f85-2258-452a-acc1-f289cecb9ec0 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:38.136784 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:38.147552 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 549/1073] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:01:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 01:01:38.148482 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4d934b-41d9-40fc-90ff-0663a00f46c7 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:38.159468 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b96cb1-6531-4d77-8653-9007d1f32d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:38.160030 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b96cb1-6531-4d77-8653-9007d1f32d6f None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:38.160579 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b96cb1-6531-4d77-8653-9007d1f32d6f None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:38.160759 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b96cb1-6531-4d77-8653-9007d1f32d6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:38.172714 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b96cb1-6531-4d77-8653-9007d1f32d6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:38.174453 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b96cb1-6531-4d77-8653-9007d1f32d6f None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:38.190607 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-b2b96cb1-6531-4d77-8653-9007d1f32d6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:01:38.190607 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:38.190607 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:38.190607 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:38.190607 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:38.190607 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:38.190607 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:38.190607 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:38.190607 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:38.190607 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:38.190607 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:38.190607 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:38.190607 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:01:38.190607 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:01:38.190607 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:01:38.190607 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:01:38.191571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:38.191571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:38.191571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:38.191571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:38.191571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:38.191571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:38.191571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:38.191571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:38.191571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:38.191571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:38.191571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:38.191571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:38.191571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:38.191571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:38.191571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:38.191571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:38.191571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:01:38.192526 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:01:38.192526 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:01:38.192526 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:38.192526 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 525/1074] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:01:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:01:38.198356 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b96cb1-6531-4d77-8653-9007d1f32d6f None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:38.249520 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:38.257637 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4d934b-41d9-40fc-90ff-0663a00f46c7 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:38.296216 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7ad31f-42f4-4d30-b14e-4e3d08c8cf75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:38.296823 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ad31f-42f4-4d30-b14e-4e3d08c8cf75 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:38.297095 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ad31f-42f4-4d30-b14e-4e3d08c8cf75 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:38.297511 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ad31f-42f4-4d30-b14e-4e3d08c8cf75 None admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:38.297988 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7ad31f-42f4-4d30-b14e-4e3d08c8cf75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:38.298289 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7ad31f-42f4-4d30-b14e-4e3d08c8cf75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:38.299642 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7ad31f-42f4-4d30-b14e-4e3d08c8cf75 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:38.300559 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b96cb1-6531-4d77-8653-9007d1f32d6f None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:38.312701 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 550/1075] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:01:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:38.318159 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7ad31f-42f4-4d30-b14e-4e3d08c8cf75 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:38.336735 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2a2f2b-1ed3-4e4b-aa0a-5c11111aa38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:38.337344 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a2f2b-1ed3-4e4b-aa0a-5c11111aa38f None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:38.337627 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a2f2b-1ed3-4e4b-aa0a-5c11111aa38f None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:38.338448 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a2f2b-1ed3-4e4b-aa0a-5c11111aa38f None admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:38.339088 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2a2f2b-1ed3-4e4b-aa0a-5c11111aa38f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:38.339391 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2a2f2b-1ed3-4e4b-aa0a-5c11111aa38f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:38.340791 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2a2f2b-1ed3-4e4b-aa0a-5c11111aa38f None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:38.364174 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 526/1076] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:01:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:38.417973 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2a2f2b-1ed3-4e4b-aa0a-5c11111aa38f None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:38.441981 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7ad31f-42f4-4d30-b14e-4e3d08c8cf75 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:38.448406 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:38.482209 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd8a88-a5f8-4165-a92b-e652175b6ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:38.482860 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd8a88-a5f8-4165-a92b-e652175b6ca6 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:38.483133 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd8a88-a5f8-4165-a92b-e652175b6ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:38.483399 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd8a88-a5f8-4165-a92b-e652175b6ca6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:38.490867 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bd8a88-a5f8-4165-a92b-e652175b6ca6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:38.492352 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bd8a88-a5f8-4165-a92b-e652175b6ca6 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:38.505050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-a9bd8a88-a5f8-4165-a92b-e652175b6ca6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:01:38.505050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:38.505050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:38.505050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:38.505050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:38.505050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:38.505050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:38.505050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:38.505050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:38.505050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:38.505050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:38.505050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:38.505050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 01:01:38.505050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 01:01:38.505050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 01:01:38.506050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 01:01:38.506050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:38.506050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:38.506050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:38.506050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:38.506050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:38.506050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:38.506050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:38.506050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:38.506050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:38.506050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:38.506050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:38.506050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:38.506050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:38.506050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:38.506050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:38.506050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:38.506951 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 01:01:38.506951 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 01:01:38.506951 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 01:01:38.506951 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 01:01:38.506951 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 01:01:38.506951 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 01:01:38.506951 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:01:38.506951 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:38.511456 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 551/1077] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 01:01:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 01:01:38.512436 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd8a88-a5f8-4165-a92b-e652175b6ca6 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:38.534768 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:38.541614 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2a2f2b-1ed3-4e4b-aa0a-5c11111aa38f None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:38.582292 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a50c2-f703-45ca-9a34-f8e3cca9fff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:38.583027 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a50c2-f703-45ca-9a34-f8e3cca9fff2 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:38.583363 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a50c2-f703-45ca-9a34-f8e3cca9fff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:38.583628 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a50c2-f703-45ca-9a34-f8e3cca9fff2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:38.598432 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31a50c2-f703-45ca-9a34-f8e3cca9fff2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:01:38.605051 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-f31a50c2-f703-45ca-9a34-f8e3cca9fff2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 01:01:38.605051 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:38.605051 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:38.605051 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:38.605051 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:38.605051 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:38.605051 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:38.605051 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:38.605051 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:38.605051 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:38.605051 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:38.605051 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:38.605051 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:01:38.605051 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:01:38.605051 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:01:38.605051 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:01:38.606225 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:01:38.606225 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:01:38.606225 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:01:38.606225 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:01:38.606225 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:38.606225 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:38.606225 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:38.606225 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:38.606225 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:38.606225 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:38.606225 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:38.606225 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:38.606225 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:38.606225 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:38.606225 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:38.606225 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:38.607166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:38.607166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:38.607166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:38.607166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:38.607166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:01:38.607166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:01:38.607166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:01:38.607166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:01:38.607166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:01:38.607166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:01:38.607166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 01:01:38.607166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:38.607166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 527/1078] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 8 01:01:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 01:01:38.612906 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a50c2-f703-45ca-9a34-f8e3cca9fff2 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:38.639503 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd8a88-a5f8-4165-a92b-e652175b6ca6 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:38.679931 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ac4ab4-b424-4c56-bdfc-4d4b54f6780f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:38.680504 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac4ab4-b424-4c56-bdfc-4d4b54f6780f None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:38.680811 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac4ab4-b424-4c56-bdfc-4d4b54f6780f None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:38.681085 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac4ab4-b424-4c56-bdfc-4d4b54f6780f None admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:38.681812 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ac4ab4-b424-4c56-bdfc-4d4b54f6780f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:38.682118 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ac4ab4-b424-4c56-bdfc-4d4b54f6780f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:38.683557 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ac4ab4-b424-4c56-bdfc-4d4b54f6780f None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:38.696188 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 552/1079] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 8 01:01:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:01:38.702107 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ac4ab4-b424-4c56-bdfc-4d4b54f6780f None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:38.711817 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:38.726786 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a50c2-f703-45ca-9a34-f8e3cca9fff2 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:38.762364 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fc3cd7-45db-4481-99e5-f586ed3b7b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:38.762970 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fc3cd7-45db-4481-99e5-f586ed3b7b62 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:38.763249 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fc3cd7-45db-4481-99e5-f586ed3b7b62 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:38.763519 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fc3cd7-45db-4481-99e5-f586ed3b7b62 None admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:38.764197 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fc3cd7-45db-4481-99e5-f586ed3b7b62 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:38.764502 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fc3cd7-45db-4481-99e5-f586ed3b7b62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:38.765875 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fc3cd7-45db-4481-99e5-f586ed3b7b62 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:38.774280 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 528/1080] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 01:01:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:38.779815 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:38.780574 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fc3cd7-45db-4481-99e5-f586ed3b7b62 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:38.802642 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ac4ab4-b424-4c56-bdfc-4d4b54f6780f None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:38.836692 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e428693-20fc-4539-b9bb-106a785109d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:38.837343 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e428693-20fc-4539-b9bb-106a785109d2 None admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:38.837618 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e428693-20fc-4539-b9bb-106a785109d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:38.838431 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e428693-20fc-4539-b9bb-106a785109d2 None admin] PATH_INFO: `/v3/projects/78ad2a8fb5264fb4af935e039d28c8de/users/ae643cfdcb164aa9b474747cd21aa116/roles/62b04068ecbd4b13a405370cb199a142` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:38.840659 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e428693-20fc-4539-b9bb-106a785109d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=78ad2a8fb5264fb4af935e039d28c8de, user_id=ae643cfdcb164aa9b474747cd21aa116, role_id=62b04068ecbd4b13a405370cb199a142)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:38.842081 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e428693-20fc-4539-b9bb-106a785109d2 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:38.854611 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 553/1081] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 8 01:01:38 2022] PUT /identity/v3/projects/78ad2a8fb5264fb4af935e039d28c8de/users/ae643cfdcb164aa9b474747cd21aa116/roles/62b04068ecbd4b13a405370cb199a142 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:01:38.880441 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fc3cd7-45db-4481-99e5-f586ed3b7b62 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:38.918997 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b12893c-a259-47b7-8487-473e239f11cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:38.919650 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12893c-a259-47b7-8487-473e239f11cc None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:38.919927 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12893c-a259-47b7-8487-473e239f11cc None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:38.920227 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12893c-a259-47b7-8487-473e239f11cc None admin] PATH_INFO: `/v3/users/f07010516657452bad5de86fc97ee8bc` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:38.921374 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b12893c-a259-47b7-8487-473e239f11cc None admin] RBAC: Authorizing `identity:get_user(user_id=f07010516657452bad5de86fc97ee8bc)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:38.922826 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b12893c-a259-47b7-8487-473e239f11cc None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:38.924474 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 529/1082] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 8 01:01:38 2022] GET /identity/v3/users/f07010516657452bad5de86fc97ee8bc => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:01:38.930508 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e428693-20fc-4539-b9bb-106a785109d2 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:39.039672 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e428693-20fc-4539-b9bb-106a785109d2 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:39.073456 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ebf626-3500-4f6e-8142-74375e045550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:39.074076 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ebf626-3500-4f6e-8142-74375e045550 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:39.074349 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ebf626-3500-4f6e-8142-74375e045550 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:39.074615 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ebf626-3500-4f6e-8142-74375e045550 None admin] PATH_INFO: `/v3/projects/b898a6e4e2da4c09893c6aa1db674bd4` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:39.075712 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ebf626-3500-4f6e-8142-74375e045550 None admin] RBAC: Authorizing `identity:get_project(project_id=b898a6e4e2da4c09893c6aa1db674bd4)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:39.077156 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ebf626-3500-4f6e-8142-74375e045550 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:39.078979 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 554/1083] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 01:01:38 2022] GET /identity/v3/projects/b898a6e4e2da4c09893c6aa1db674bd4 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:01:39.084629 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b12893c-a259-47b7-8487-473e239f11cc None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:39.103490 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:39.189719 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b12893c-a259-47b7-8487-473e239f11cc None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:39.224071 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a3fe05-8e5f-4e52-9c70-251fb3f32af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:39.224747 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3fe05-8e5f-4e52-9c70-251fb3f32af7 None admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:39.225025 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3fe05-8e5f-4e52-9c70-251fb3f32af7 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:39.225294 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3fe05-8e5f-4e52-9c70-251fb3f32af7 None admin] PATH_INFO: `/v3/projects/b898a6e4e2da4c09893c6aa1db674bd4/users/f07010516657452bad5de86fc97ee8bc/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:39.227162 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a3fe05-8e5f-4e52-9c70-251fb3f32af7 None admin] RBAC: Authorizing `identity:create_grant(project_id=b898a6e4e2da4c09893c6aa1db674bd4, user_id=f07010516657452bad5de86fc97ee8bc, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:39.228759 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a3fe05-8e5f-4e52-9c70-251fb3f32af7 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:39.241523 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 530/1084] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 8 01:01:39 2022] PUT /identity/v3/projects/b898a6e4e2da4c09893c6aa1db674bd4/users/f07010516657452bad5de86fc97ee8bc/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:01:41.022006 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd62c3-dcc6-43d2-b093-4d7958a1c5a5 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:41.022554 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd62c3-dcc6-43d2-b093-4d7958a1c5a5 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:41.022554 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd62c3-dcc6-43d2-b093-4d7958a1c5a5 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:41.023889 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 555/1085] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:01:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:01:41.029929 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f97cc7-d404-4021-bf12-e333a2c79933 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:41.030328 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f97cc7-d404-4021-bf12-e333a2c79933 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:41.030487 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f97cc7-d404-4021-bf12-e333a2c79933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:41.048494 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-13f97cc7-d404-4021-bf12-e333a2c79933 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:01:41.049687 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-13f97cc7-d404-4021-bf12-e333a2c79933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:01:41.115286 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 531/1086] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:01:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 01:01:41.369488 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192b6e8-5f8d-4b06-86ed-d83076492b57 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:41.369795 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192b6e8-5f8d-4b06-86ed-d83076492b57 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:41.370069 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192b6e8-5f8d-4b06-86ed-d83076492b57 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:41.371406 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 556/1087] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:01:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:01:41.376663 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a59c3c-11d0-4086-b5dc-8fa9e736ad2e None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:41.376944 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a59c3c-11d0-4086-b5dc-8fa9e736ad2e None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:41.377215 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a59c3c-11d0-4086-b5dc-8fa9e736ad2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:41.384335 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3192b6e8-5f8d-4b06-86ed-d83076492b57 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:41.389097 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:41.390544 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:41.397305 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-09a59c3c-11d0-4086-b5dc-8fa9e736ad2e None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:01:41.398666 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-09a59c3c-11d0-4086-b5dc-8fa9e736ad2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:01:41.467069 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 532/1088] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:01:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 01:01:41.487971 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3192b6e8-5f8d-4b06-86ed-d83076492b57 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:41.521841 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b9a98a-8df9-4080-87f1-c883c1451774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:41.522456 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9a98a-8df9-4080-87f1-c883c1451774 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:41.522730 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9a98a-8df9-4080-87f1-c883c1451774 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:41.522998 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9a98a-8df9-4080-87f1-c883c1451774 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:41.532866 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b9a98a-8df9-4080-87f1-c883c1451774 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:01:41.537819 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-60b9a98a-8df9-4080-87f1-c883c1451774 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 01:01:41.537819 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:41.537819 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:41.537819 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:41.537819 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:41.537819 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:41.537819 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:41.537819 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:41.537819 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:41.537819 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:41.537819 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:41.537819 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:41.537819 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:01:41.537819 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:01:41.537819 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:01:41.537819 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:01:41.538759 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:01:41.538759 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:01:41.538759 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:01:41.538759 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:01:41.538759 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:41.538759 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:41.538759 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:41.538759 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:41.538759 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:41.538759 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:41.538759 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:41.538759 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:41.538759 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:41.538759 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:41.538759 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:41.538759 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:41.539618 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:41.539618 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:41.539618 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:41.539618 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:41.539618 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:01:41.539618 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:01:41.539618 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:01:41.539618 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:01:41.539618 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:01:41.539618 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:01:41.539618 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 01:01:41.539618 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:41.539618 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 557/1089] 10.222.0.42 () {56 vars in 1120 bytes} [Thu Sep 8 01:01:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 01:01:41.544886 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a59c3c-11d0-4086-b5dc-8fa9e736ad2e None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:41.644014 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a59c3c-11d0-4086-b5dc-8fa9e736ad2e None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:41.676497 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd203bc6-7ff6-4a88-b9e1-4125f54a331a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:41.677090 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd203bc6-7ff6-4a88-b9e1-4125f54a331a None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:41.677362 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd203bc6-7ff6-4a88-b9e1-4125f54a331a None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:41.677627 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd203bc6-7ff6-4a88-b9e1-4125f54a331a None admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:41.678282 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd203bc6-7ff6-4a88-b9e1-4125f54a331a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:41.678577 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd203bc6-7ff6-4a88-b9e1-4125f54a331a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:41.679962 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd203bc6-7ff6-4a88-b9e1-4125f54a331a None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:41.682706 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:41.689242 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 533/1090] 10.222.0.42 () {56 vars in 1115 bytes} [Thu Sep 8 01:01:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:41.695049 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b9a98a-8df9-4080-87f1-c883c1451774 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:41.748275 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd203bc6-7ff6-4a88-b9e1-4125f54a331a None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:41.755554 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:41.794153 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b9a98a-8df9-4080-87f1-c883c1451774 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:41.827727 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea424720-f460-4a40-87ac-bd7d95ce6a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:41.828372 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea424720-f460-4a40-87ac-bd7d95ce6a99 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:41.828649 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea424720-f460-4a40-87ac-bd7d95ce6a99 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:41.828922 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea424720-f460-4a40-87ac-bd7d95ce6a99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:41.839151 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea424720-f460-4a40-87ac-bd7d95ce6a99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:41.840638 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea424720-f460-4a40-87ac-bd7d95ce6a99 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:41.846781 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd203bc6-7ff6-4a88-b9e1-4125f54a331a None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:41.855221 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-ea424720-f460-4a40-87ac-bd7d95ce6a99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:01:41.855221 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:41.855221 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:41.855221 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:41.855221 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:41.855221 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:41.855221 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:41.855221 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:41.855221 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:41.855221 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:41.855221 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:41.855221 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:41.855221 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:01:41.855221 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:01:41.855221 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:01:41.855221 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:01:41.856285 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:41.856285 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:41.856285 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:41.856285 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:41.856285 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:41.856285 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:41.856285 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:41.856285 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:41.856285 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:41.856285 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:41.856285 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:41.856285 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:41.856285 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:41.856285 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:41.856285 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:41.856285 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:41.856285 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:01:41.857700 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:01:41.857700 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:01:41.857700 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:41.857700 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 558/1091] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:01:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 01:01:41.861900 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea424720-f460-4a40-87ac-bd7d95ce6a99 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:41.883805 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a60277-e580-42ce-bf09-c6cdead5e09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:41.884382 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a60277-e580-42ce-bf09-c6cdead5e09f None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:41.885795 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a60277-e580-42ce-bf09-c6cdead5e09f None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:41.885795 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a60277-e580-42ce-bf09-c6cdead5e09f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:41.896027 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a60277-e580-42ce-bf09-c6cdead5e09f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:01:41.901148 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-d6a60277-e580-42ce-bf09-c6cdead5e09f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 01:01:41.901148 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:41.901148 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:41.901148 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:41.901148 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:41.901148 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:41.901148 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:41.901148 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:41.901148 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:41.901148 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:41.901148 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:41.901148 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:41.901148 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:01:41.901148 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:01:41.901148 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:01:41.901148 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:01:41.902166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:01:41.902166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:01:41.902166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:01:41.902166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:01:41.902166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:41.902166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:41.902166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:41.902166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:41.902166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:41.902166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:41.902166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:41.902166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:41.902166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:41.902166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:41.902166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:41.902166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:41.903001 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:41.903001 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:41.903001 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:41.903001 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:41.903001 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:01:41.903001 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:01:41.903001 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:01:41.903001 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:01:41.903001 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:01:41.903001 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:01:41.903001 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 01:01:41.903001 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:01:41.903001 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 534/1092] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 8 01:01:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 01:01:41.908219 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a60277-e580-42ce-bf09-c6cdead5e09f None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:41.961781 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea424720-f460-4a40-87ac-bd7d95ce6a99 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:41.965238 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:41.977531 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:41.997283 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45fd256-b5e5-4e3f-a6ce-fb3615c3e0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:41.997875 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45fd256-b5e5-4e3f-a6ce-fb3615c3e0c9 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:41.998148 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45fd256-b5e5-4e3f-a6ce-fb3615c3e0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:41.998415 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45fd256-b5e5-4e3f-a6ce-fb3615c3e0c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:41.999050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45fd256-b5e5-4e3f-a6ce-fb3615c3e0c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:41.999347 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45fd256-b5e5-4e3f-a6ce-fb3615c3e0c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:42.000740 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45fd256-b5e5-4e3f-a6ce-fb3615c3e0c9 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:42.008072 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a60277-e580-42ce-bf09-c6cdead5e09f None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:42.012291 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 559/1093] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:01:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:01:42.017853 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45fd256-b5e5-4e3f-a6ce-fb3615c3e0c9 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:42.042215 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c6cb2a-59ab-4d3d-be47-469f7accf2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:42.042819 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c6cb2a-59ab-4d3d-be47-469f7accf2a4 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:42.043100 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c6cb2a-59ab-4d3d-be47-469f7accf2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:42.043375 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c6cb2a-59ab-4d3d-be47-469f7accf2a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:42.044035 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c6cb2a-59ab-4d3d-be47-469f7accf2a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:42.044370 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c6cb2a-59ab-4d3d-be47-469f7accf2a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:42.045744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c6cb2a-59ab-4d3d-be47-469f7accf2a4 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:42.054027 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 535/1094] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 01:01:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:01:42.060179 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c6cb2a-59ab-4d3d-be47-469f7accf2a4 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:42.118106 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45fd256-b5e5-4e3f-a6ce-fb3615c3e0c9 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:42.153637 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5354766-827a-4f5c-8d33-e83fb3b505da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:42.154188 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5354766-827a-4f5c-8d33-e83fb3b505da None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:42.154468 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5354766-827a-4f5c-8d33-e83fb3b505da None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:42.154739 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5354766-827a-4f5c-8d33-e83fb3b505da None admin] PATH_INFO: `/v3/users/nova` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:42.157986 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c6cb2a-59ab-4d3d-be47-469f7accf2a4 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:42.161886 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5354766-827a-4f5c-8d33-e83fb3b505da None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:42.163318 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5354766-827a-4f5c-8d33-e83fb3b505da None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:42.175838 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-a5354766-827a-4f5c-8d33-e83fb3b505da None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 01:01:42.175838 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:42.175838 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:42.175838 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:42.175838 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:42.175838 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:42.175838 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:42.175838 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:42.175838 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:42.175838 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:42.175838 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:42.175838 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:42.175838 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 01:01:42.175838 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 01:01:42.175838 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 01:01:42.175838 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 01:01:42.176847 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:42.176847 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:42.176847 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 01:01:42.176847 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:01:42.176847 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 01:01:42.176847 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:01:42.176847 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:42.176847 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:42.176847 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:42.176847 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:42.176847 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:42.176847 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:42.176847 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:42.176847 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:42.176847 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:42.176847 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:42.176847 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:42.176847 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:42.177778 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:42.177778 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:42.177778 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 01:01:42.177778 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 01:01:42.177778 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 01:01:42.177778 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 01:01:42.177778 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 01:01:42.177778 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 01:01:42.177778 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 01:01:42.177778 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:42.177778 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 560/1095] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Sep 8 01:01:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 01:01:42.178366 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:42.182909 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5354766-827a-4f5c-8d33-e83fb3b505da None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:42.195192 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d125131c-22e7-433a-abae-d06f808d2e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:42.195831 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d125131c-22e7-433a-abae-d06f808d2e46 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:42.196112 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d125131c-22e7-433a-abae-d06f808d2e46 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:42.196412 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d125131c-22e7-433a-abae-d06f808d2e46 None admin] PATH_INFO: `/v3/users/f07010516657452bad5de86fc97ee8bc` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:42.197534 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d125131c-22e7-433a-abae-d06f808d2e46 None admin] RBAC: Authorizing `identity:get_user(user_id=f07010516657452bad5de86fc97ee8bc)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:42.199065 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d125131c-22e7-433a-abae-d06f808d2e46 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:42.200741 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 536/1096] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 8 01:01:42 2022] GET /identity/v3/users/f07010516657452bad5de86fc97ee8bc => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:01:42.206267 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d125131c-22e7-433a-abae-d06f808d2e46 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:42.219288 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:42.284848 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5354766-827a-4f5c-8d33-e83fb3b505da None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:42.304530 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d125131c-22e7-433a-abae-d06f808d2e46 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:42.318096 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb2a81-90e9-4226-8831-aedde4b6806a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:42.318689 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb2a81-90e9-4226-8831-aedde4b6806a None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:42.318962 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb2a81-90e9-4226-8831-aedde4b6806a None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:42.319227 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb2a81-90e9-4226-8831-aedde4b6806a None admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:42.319943 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cb2a81-90e9-4226-8831-aedde4b6806a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:42.320275 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cb2a81-90e9-4226-8831-aedde4b6806a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:42.321685 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cb2a81-90e9-4226-8831-aedde4b6806a None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:42.338776 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 561/1097] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Sep 8 01:01:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:01:42.342178 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20605244-8ae8-4d69-98e6-fe2f8ce3e808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:42.342782 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20605244-8ae8-4d69-98e6-fe2f8ce3e808 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:42.343060 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20605244-8ae8-4d69-98e6-fe2f8ce3e808 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:42.343329 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20605244-8ae8-4d69-98e6-fe2f8ce3e808 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:42.344553 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb2a81-90e9-4226-8831-aedde4b6806a None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:42.399140 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:42.412737 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20605244-8ae8-4d69-98e6-fe2f8ce3e808 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:42.414040 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20605244-8ae8-4d69-98e6-fe2f8ce3e808 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:42.436525 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:42.443546 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb2a81-90e9-4226-8831-aedde4b6806a None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:42.477784 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3219ae93-a670-4d49-a6ca-d5a42fd57d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:42.478954 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219ae93-a670-4d49-a6ca-d5a42fd57d05 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:42.479231 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219ae93-a670-4d49-a6ca-d5a42fd57d05 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:42.479499 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219ae93-a670-4d49-a6ca-d5a42fd57d05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:42.512559 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 537/1098] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 01:01:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 08 01:01:42.518479 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20605244-8ae8-4d69-98e6-fe2f8ce3e808 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:42.547551 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3219ae93-a670-4d49-a6ca-d5a42fd57d05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:42.549030 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3219ae93-a670-4d49-a6ca-d5a42fd57d05 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:42.602202 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:42.614588 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20605244-8ae8-4d69-98e6-fe2f8ce3e808 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:42.642431 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:42.650715 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 562/1099] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 01:01:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 08 01:01:42.650961 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ac1949-f92d-4856-9848-1d748605df62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:42.651589 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ac1949-f92d-4856-9848-1d748605df62 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:42.651892 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ac1949-f92d-4856-9848-1d748605df62 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:42.652186 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ac1949-f92d-4856-9848-1d748605df62 None admin] PATH_INFO: `/v3/projects/b898a6e4e2da4c09893c6aa1db674bd4` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:42.653301 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ac1949-f92d-4856-9848-1d748605df62 None admin] RBAC: Authorizing `identity:get_project(project_id=b898a6e4e2da4c09893c6aa1db674bd4)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:42.654729 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ac1949-f92d-4856-9848-1d748605df62 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:42.656368 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 538/1100] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 01:01:42 2022] GET /identity/v3/projects/b898a6e4e2da4c09893c6aa1db674bd4 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:01:42.660228 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3219ae93-a670-4d49-a6ca-d5a42fd57d05 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:42.662198 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ac1949-f92d-4856-9848-1d748605df62 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:42.757809 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3219ae93-a670-4d49-a6ca-d5a42fd57d05 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:42.759141 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ac1949-f92d-4856-9848-1d748605df62 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:42.793420 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca30de4-f8c0-4cbf-9df9-766c6f1b2ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:42.793644 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc58a278-63a5-4458-8f9c-1d16baa09bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:42.794014 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca30de4-f8c0-4cbf-9df9-766c6f1b2ac7 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:42.794124 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc58a278-63a5-4458-8f9c-1d16baa09bbc None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:42.794289 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca30de4-f8c0-4cbf-9df9-766c6f1b2ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:42.794391 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc58a278-63a5-4458-8f9c-1d16baa09bbc None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:42.794553 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca30de4-f8c0-4cbf-9df9-766c6f1b2ac7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:42.794731 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc58a278-63a5-4458-8f9c-1d16baa09bbc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:42.795269 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca30de4-f8c0-4cbf-9df9-766c6f1b2ac7 None admin] RBAC: Adding query filter params (role.id=34de369ec7c54715aa83a9052a26c3f2, scope.project.id=b898a6e4e2da4c09893c6aa1db674bd4, user.id=f07010516657452bad5de86fc97ee8bc) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:42.795572 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca30de4-f8c0-4cbf-9df9-766c6f1b2ac7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:42.797028 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca30de4-f8c0-4cbf-9df9-766c6f1b2ac7 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:42.802644 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 539/1101] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 8 01:01:42 2022] GET /identity/v3/role_assignments?user.id=f07010516657452bad5de86fc97ee8bc&scope.project.id=b898a6e4e2da4c09893c6aa1db674bd4&role.id=34de369ec7c54715aa83a9052a26c3f2 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:01:42.810454 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc58a278-63a5-4458-8f9c-1d16baa09bbc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:42.811943 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc58a278-63a5-4458-8f9c-1d16baa09bbc None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:42.829902 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-bc58a278-63a5-4458-8f9c-1d16baa09bbc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:01:42.829902 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:42.829902 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:42.829902 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:42.829902 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:42.829902 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:42.829902 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:42.829902 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:42.829902 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:42.829902 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:42.829902 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:42.829902 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:42.829902 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:01:42.829902 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:01:42.829902 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:01:42.829902 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:01:42.831126 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:42.831126 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:42.831126 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:42.831126 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:42.831126 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:42.831126 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:42.831126 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:42.831126 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:42.831126 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:42.831126 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:42.831126 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:42.831126 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:42.831126 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:42.831126 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:42.831126 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:42.831126 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:42.831126 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:01:42.832027 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:01:42.832027 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:01:42.832027 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:42.832027 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 563/1102] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:01:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:01:42.837246 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca30de4-f8c0-4cbf-9df9-766c6f1b2ac7 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:42.860878 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:42.945390 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca30de4-f8c0-4cbf-9df9-766c6f1b2ac7 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:42.980765 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3383fd6d-c50b-409d-bc18-a4e96c02642d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:42.981384 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3383fd6d-c50b-409d-bc18-a4e96c02642d None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:42.981664 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3383fd6d-c50b-409d-bc18-a4e96c02642d None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:42.981941 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3383fd6d-c50b-409d-bc18-a4e96c02642d None admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:42.982969 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3383fd6d-c50b-409d-bc18-a4e96c02642d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:42.982969 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3383fd6d-c50b-409d-bc18-a4e96c02642d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:42.984355 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3383fd6d-c50b-409d-bc18-a4e96c02642d None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:42.996757 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 540/1103] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:01:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:43.002727 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc58a278-63a5-4458-8f9c-1d16baa09bbc None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:43.051098 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:43.104772 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc58a278-63a5-4458-8f9c-1d16baa09bbc None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:43.137314 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dad627-cc61-47bd-a428-dbdfcdd947eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:43.137941 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dad627-cc61-47bd-a428-dbdfcdd947eb None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:43.138195 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dad627-cc61-47bd-a428-dbdfcdd947eb None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:43.138468 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dad627-cc61-47bd-a428-dbdfcdd947eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:43.145418 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dad627-cc61-47bd-a428-dbdfcdd947eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:43.147191 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dad627-cc61-47bd-a428-dbdfcdd947eb None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:43.159651 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-07dad627-cc61-47bd-a428-dbdfcdd947eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:01:43.159651 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:01:43.159651 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:01:43.159651 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:01:43.159651 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:01:43.159651 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:01:43.159651 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:01:43.159651 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:01:43.159651 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:01:43.159651 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:01:43.159651 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:01:43.159651 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:01:43.159651 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 01:01:43.159651 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 01:01:43.159651 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 01:01:43.160860 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 01:01:43.160860 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:01:43.160860 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:01:43.160860 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:01:43.160860 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:01:43.160860 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:01:43.160860 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:01:43.160860 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:01:43.160860 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:01:43.160860 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:01:43.160860 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:01:43.160860 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:01:43.160860 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:01:43.160860 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:01:43.160860 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:01:43.160860 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:01:43.160860 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:01:43.162189 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 01:01:43.162189 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 01:01:43.162189 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 01:01:43.162189 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 01:01:43.162189 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 01:01:43.162189 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 01:01:43.162189 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:01:43.162189 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:01:43.162189 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 564/1104] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 01:01:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:01:43.167320 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3383fd6d-c50b-409d-bc18-a4e96c02642d None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:43.246930 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:43.266669 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3383fd6d-c50b-409d-bc18-a4e96c02642d None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:43.300524 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddbaf08-368f-428f-bd75-ec2aec8554d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:43.301188 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddbaf08-368f-428f-bd75-ec2aec8554d3 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:43.301408 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddbaf08-368f-428f-bd75-ec2aec8554d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:43.301681 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddbaf08-368f-428f-bd75-ec2aec8554d3 None admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:43.302366 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddbaf08-368f-428f-bd75-ec2aec8554d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:43.302670 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddbaf08-368f-428f-bd75-ec2aec8554d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:43.304113 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddbaf08-368f-428f-bd75-ec2aec8554d3 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:43.316367 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 541/1105] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 8 01:01:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:43.322175 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dad627-cc61-47bd-a428-dbdfcdd947eb None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:43.419641 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dad627-cc61-47bd-a428-dbdfcdd947eb None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:43.426404 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:43.454167 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931c422b-23cb-4f1b-9caa-3ad74dc53661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:43.454755 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c422b-23cb-4f1b-9caa-3ad74dc53661 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:43.455046 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c422b-23cb-4f1b-9caa-3ad74dc53661 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:43.455295 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c422b-23cb-4f1b-9caa-3ad74dc53661 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:43.456004 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931c422b-23cb-4f1b-9caa-3ad74dc53661 None admin] RBAC: Adding query filter params (role.id=62b04068ecbd4b13a405370cb199a142, scope.project.id=78ad2a8fb5264fb4af935e039d28c8de, user.id=ae643cfdcb164aa9b474747cd21aa116) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:43.456338 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931c422b-23cb-4f1b-9caa-3ad74dc53661 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:43.457749 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931c422b-23cb-4f1b-9caa-3ad74dc53661 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:43.463513 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 565/1106] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 8 01:01:43 2022] GET /identity/v3/role_assignments?user.id=ae643cfdcb164aa9b474747cd21aa116&scope.project.id=78ad2a8fb5264fb4af935e039d28c8de&role.id=62b04068ecbd4b13a405370cb199a142 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:45.000124 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b17f5fb-069b-4c2c-bd6c-acac5d9a81ec None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:45.000799 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b17f5fb-069b-4c2c-bd6c-acac5d9a81ec None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:45.000799 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b17f5fb-069b-4c2c-bd6c-acac5d9a81ec None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:45.002047 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 542/1107] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:01:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:01:45.007237 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293261fb-3775-4a63-a117-118e4ec1a974 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:45.007669 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293261fb-3775-4a63-a117-118e4ec1a974 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:45.007787 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293261fb-3775-4a63-a117-118e4ec1a974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:45.027116 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-293261fb-3775-4a63-a117-118e4ec1a974 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:01:45.028326 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-293261fb-3775-4a63-a117-118e4ec1a974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:01:45.098140 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 566/1108] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:01:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 01:01:45.372170 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b17f5fb-069b-4c2c-bd6c-acac5d9a81ec None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:45.472347 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b17f5fb-069b-4c2c-bd6c-acac5d9a81ec None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:45.504947 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2170a1c-f262-4a9b-926a-65e606ef1a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:45.505557 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2170a1c-f262-4a9b-926a-65e606ef1a09 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:45.505835 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2170a1c-f262-4a9b-926a-65e606ef1a09 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:45.506106 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2170a1c-f262-4a9b-926a-65e606ef1a09 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:45.507198 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2170a1c-f262-4a9b-926a-65e606ef1a09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:45.508729 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2170a1c-f262-4a9b-926a-65e606ef1a09 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:45.510361 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 543/1109] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:01:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:45.516496 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293261fb-3775-4a63-a117-118e4ec1a974 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:45.609877 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:45.612587 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293261fb-3775-4a63-a117-118e4ec1a974 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:45.647009 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43b97c6-50be-4c45-b637-4b6a3a754a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:45.647615 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b97c6-50be-4c45-b637-4b6a3a754a87 None admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:45.647891 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b97c6-50be-4c45-b637-4b6a3a754a87 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:45.648180 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b97c6-50be-4c45-b637-4b6a3a754a87 None admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:45.648823 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43b97c6-50be-4c45-b637-4b6a3a754a87 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:45.650457 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43b97c6-50be-4c45-b637-4b6a3a754a87 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:45.671196 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 567/1110] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 8 01:01:45 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:01:47.827462 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2740b79-cc98-4dfc-a646-51acff97f716 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:47.828013 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2740b79-cc98-4dfc-a646-51acff97f716 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:47.828013 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2740b79-cc98-4dfc-a646-51acff97f716 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:47.829356 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 544/1111] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:01:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:01:47.834543 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8397d88-f588-4800-b567-4eb79fbe948d None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:47.834908 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8397d88-f588-4800-b567-4eb79fbe948d None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:47.835099 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8397d88-f588-4800-b567-4eb79fbe948d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:47.853665 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-c8397d88-f588-4800-b567-4eb79fbe948d None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:01:47.854872 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-c8397d88-f588-4800-b567-4eb79fbe948d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:01:47.920030 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 568/1112] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:01:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 01:01:48.197572 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2740b79-cc98-4dfc-a646-51acff97f716 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:48.208711 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:48.298995 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2740b79-cc98-4dfc-a646-51acff97f716 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:48.331675 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbce214-0efd-44c1-a38f-c5ed0af8fa6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:48.332317 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbce214-0efd-44c1-a38f-c5ed0af8fa6d None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:48.332594 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbce214-0efd-44c1-a38f-c5ed0af8fa6d None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:48.332864 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbce214-0efd-44c1-a38f-c5ed0af8fa6d None admin] PATH_INFO: `/v3/roles/069fe3e033664493b3def844e3735bc9` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:48.334217 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbce214-0efd-44c1-a38f-c5ed0af8fa6d None admin] RBAC: Authorizing `identity:get_role(role_id=069fe3e033664493b3def844e3735bc9)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:48.335584 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbce214-0efd-44c1-a38f-c5ed0af8fa6d None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:48.336922 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 545/1113] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 8 01:01:48 2022] GET /identity/v3/roles/069fe3e033664493b3def844e3735bc9 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:48.343228 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8397d88-f588-4800-b567-4eb79fbe948d None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:48.426237 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:48.441601 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8397d88-f588-4800-b567-4eb79fbe948d None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:48.474522 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb54fc2-3643-4485-8131-954307994013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:48.475075 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb54fc2-3643-4485-8131-954307994013 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:48.475345 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb54fc2-3643-4485-8131-954307994013 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:48.475611 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb54fc2-3643-4485-8131-954307994013 None admin] PATH_INFO: `/v3/users/b3a765997bc54d43991214bfaaf30437` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:48.488626 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb54fc2-3643-4485-8131-954307994013 None admin] RBAC: Authorizing `identity:get_user(user_id=b3a765997bc54d43991214bfaaf30437)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:48.490045 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb54fc2-3643-4485-8131-954307994013 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:48.491600 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 569/1114] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 8 01:01:48 2022] GET /identity/v3/users/b3a765997bc54d43991214bfaaf30437 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:48.498004 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbce214-0efd-44c1-a38f-c5ed0af8fa6d None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:48.572931 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:48.595832 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbce214-0efd-44c1-a38f-c5ed0af8fa6d None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:48.628263 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d576f0-f687-4496-82c0-d3f47f147c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:48.628858 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d576f0-f687-4496-82c0-d3f47f147c32 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:48.629133 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d576f0-f687-4496-82c0-d3f47f147c32 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:48.629399 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d576f0-f687-4496-82c0-d3f47f147c32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:48.695329 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d576f0-f687-4496-82c0-d3f47f147c32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:48.696795 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d576f0-f687-4496-82c0-d3f47f147c32 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:48.769052 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:48.792261 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 546/1115] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 01:01:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 01:01:48.798107 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb54fc2-3643-4485-8131-954307994013 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:48.893443 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb54fc2-3643-4485-8131-954307994013 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:48.926109 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc77e7c-c310-4ad2-8628-eb253cc8b5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:48.926727 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc77e7c-c310-4ad2-8628-eb253cc8b5cd None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:48.927007 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc77e7c-c310-4ad2-8628-eb253cc8b5cd None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:48.927285 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc77e7c-c310-4ad2-8628-eb253cc8b5cd None admin] PATH_INFO: `/v3/projects/b898a6e4e2da4c09893c6aa1db674bd4` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:48.928420 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc77e7c-c310-4ad2-8628-eb253cc8b5cd None admin] RBAC: Authorizing `identity:get_project(project_id=b898a6e4e2da4c09893c6aa1db674bd4)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:48.929850 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc77e7c-c310-4ad2-8628-eb253cc8b5cd None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:48.932191 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 570/1116] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 01:01:48 2022] GET /identity/v3/projects/b898a6e4e2da4c09893c6aa1db674bd4 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:48.938112 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d576f0-f687-4496-82c0-d3f47f147c32 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:49.033683 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d576f0-f687-4496-82c0-d3f47f147c32 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:49.066509 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef325fdd-b623-4dd9-add4-e7d1e9a7392d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:49.067102 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef325fdd-b623-4dd9-add4-e7d1e9a7392d None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:49.067383 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef325fdd-b623-4dd9-add4-e7d1e9a7392d None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:49.067653 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef325fdd-b623-4dd9-add4-e7d1e9a7392d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:49.068397 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef325fdd-b623-4dd9-add4-e7d1e9a7392d None admin] RBAC: Adding query filter params (role.id=069fe3e033664493b3def844e3735bc9, scope.project.id=b898a6e4e2da4c09893c6aa1db674bd4, user.id=b3a765997bc54d43991214bfaaf30437) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:49.068707 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef325fdd-b623-4dd9-add4-e7d1e9a7392d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:49.070128 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef325fdd-b623-4dd9-add4-e7d1e9a7392d None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:49.075859 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 547/1117] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 8 01:01:48 2022] GET /identity/v3/role_assignments?user.id=b3a765997bc54d43991214bfaaf30437&scope.project.id=b898a6e4e2da4c09893c6aa1db674bd4&role.id=069fe3e033664493b3def844e3735bc9 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:51.213466 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c670a-51cb-442e-95da-8fb306550b28 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:51.214004 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c670a-51cb-442e-95da-8fb306550b28 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:51.214098 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c670a-51cb-442e-95da-8fb306550b28 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:51.215553 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 571/1118] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:01:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:01:51.220880 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b896c7-0f09-4f94-884b-1a267613075f None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:51.221319 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b896c7-0f09-4f94-884b-1a267613075f None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:51.221422 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b896c7-0f09-4f94-884b-1a267613075f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:51.239780 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-f2b896c7-0f09-4f94-884b-1a267613075f None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:01:51.241007 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-f2b896c7-0f09-4f94-884b-1a267613075f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:01:51.273982 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:51.307887 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 548/1119] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:01:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 01:01:51.577906 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9c670a-51cb-442e-95da-8fb306550b28 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:51.592668 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:51.677721 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9c670a-51cb-442e-95da-8fb306550b28 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:51.710534 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6ad6f1-e937-461f-8248-90b652d001d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:51.711147 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ad6f1-e937-461f-8248-90b652d001d7 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:51.711424 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ad6f1-e937-461f-8248-90b652d001d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:51.711694 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ad6f1-e937-461f-8248-90b652d001d7 None admin] PATH_INFO: `/v3/roles/069fe3e033664493b3def844e3735bc9` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:51.713125 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6ad6f1-e937-461f-8248-90b652d001d7 None admin] RBAC: Authorizing `identity:get_role(role_id=069fe3e033664493b3def844e3735bc9)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:51.714479 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6ad6f1-e937-461f-8248-90b652d001d7 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:51.715785 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 572/1120] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 8 01:01:51 2022] GET /identity/v3/roles/069fe3e033664493b3def844e3735bc9 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:51.721588 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b896c7-0f09-4f94-884b-1a267613075f None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:51.819067 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b896c7-0f09-4f94-884b-1a267613075f None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:51.851400 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4904ae7d-caa7-4077-b27a-8c384946b574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:51.851961 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4904ae7d-caa7-4077-b27a-8c384946b574 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:51.852271 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4904ae7d-caa7-4077-b27a-8c384946b574 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:51.852550 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4904ae7d-caa7-4077-b27a-8c384946b574 None admin] PATH_INFO: `/v3/users/b3a765997bc54d43991214bfaaf30437` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:51.854307 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4904ae7d-caa7-4077-b27a-8c384946b574 None admin] RBAC: Authorizing `identity:get_user(user_id=b3a765997bc54d43991214bfaaf30437)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:51.855740 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4904ae7d-caa7-4077-b27a-8c384946b574 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:51.857335 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 549/1121] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 8 01:01:51 2022] GET /identity/v3/users/b3a765997bc54d43991214bfaaf30437 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:51.862924 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6ad6f1-e937-461f-8248-90b652d001d7 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:51.941281 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:51.960071 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6ad6f1-e937-461f-8248-90b652d001d7 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:51.993279 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ebbe5b-5934-4411-ab61-226e1b111b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:51.993941 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ebbe5b-5934-4411-ab61-226e1b111b45 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:51.994215 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ebbe5b-5934-4411-ab61-226e1b111b45 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:51.994485 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ebbe5b-5934-4411-ab61-226e1b111b45 None admin] PATH_INFO: `/v3/projects/b898a6e4e2da4c09893c6aa1db674bd4` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:51.995597 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ebbe5b-5934-4411-ab61-226e1b111b45 None admin] RBAC: Authorizing `identity:get_project(project_id=b898a6e4e2da4c09893c6aa1db674bd4)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:51.997041 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ebbe5b-5934-4411-ab61-226e1b111b45 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:51.998969 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 573/1122] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 01:01:51 2022] GET /identity/v3/projects/b898a6e4e2da4c09893c6aa1db674bd4 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:52.004764 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4904ae7d-caa7-4077-b27a-8c384946b574 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:52.044605 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:52.102419 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4904ae7d-caa7-4077-b27a-8c384946b574 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:52.135349 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eb2b68-4fd1-45e7-abcf-55d9abfc5810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:52.135984 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eb2b68-4fd1-45e7-abcf-55d9abfc5810 None admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:52.136557 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eb2b68-4fd1-45e7-abcf-55d9abfc5810 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:52.136827 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eb2b68-4fd1-45e7-abcf-55d9abfc5810 None admin] PATH_INFO: `/v3/projects/b898a6e4e2da4c09893c6aa1db674bd4/users/b3a765997bc54d43991214bfaaf30437/roles/069fe3e033664493b3def844e3735bc9` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:52.138907 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eb2b68-4fd1-45e7-abcf-55d9abfc5810 None admin] RBAC: Authorizing `identity:create_grant(project_id=b898a6e4e2da4c09893c6aa1db674bd4, user_id=b3a765997bc54d43991214bfaaf30437, role_id=069fe3e033664493b3def844e3735bc9)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:52.140352 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eb2b68-4fd1-45e7-abcf-55d9abfc5810 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:52.153655 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 550/1123] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 8 01:01:52 2022] PUT /identity/v3/projects/b898a6e4e2da4c09893c6aa1db674bd4/users/b3a765997bc54d43991214bfaaf30437/roles/069fe3e033664493b3def844e3735bc9 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:01:54.277745 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d921b-2c80-4a75-9376-95bf9ee03735 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:54.278275 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d921b-2c80-4a75-9376-95bf9ee03735 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:54.278275 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d921b-2c80-4a75-9376-95bf9ee03735 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:54.279603 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 574/1124] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:01:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:01:54.284945 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da485fa-3aec-4998-919d-e0e21c8a4e5b None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:54.285345 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da485fa-3aec-4998-919d-e0e21c8a4e5b None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:54.285495 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da485fa-3aec-4998-919d-e0e21c8a4e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:54.303730 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-3da485fa-3aec-4998-919d-e0e21c8a4e5b None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:01:54.304950 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-3da485fa-3aec-4998-919d-e0e21c8a4e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:01:54.369073 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 551/1125] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:01:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 01:01:54.637663 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9d921b-2c80-4a75-9376-95bf9ee03735 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:54.738369 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9d921b-2c80-4a75-9376-95bf9ee03735 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:54.772058 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc0981-38e3-4603-a593-870178357d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:54.772932 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc0981-38e3-4603-a593-870178357d77 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:54.773219 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc0981-38e3-4603-a593-870178357d77 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:54.773494 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc0981-38e3-4603-a593-870178357d77 None admin] PATH_INFO: `/v3/roles/069fe3e033664493b3def844e3735bc9` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:54.774968 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcc0981-38e3-4603-a593-870178357d77 None admin] RBAC: Authorizing `identity:get_role(role_id=069fe3e033664493b3def844e3735bc9)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:54.776484 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcc0981-38e3-4603-a593-870178357d77 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:54.777811 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 575/1126] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 8 01:01:54 2022] GET /identity/v3/roles/069fe3e033664493b3def844e3735bc9 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:54.783725 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da485fa-3aec-4998-919d-e0e21c8a4e5b None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:54.811547 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:54.881837 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da485fa-3aec-4998-919d-e0e21c8a4e5b None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:54.914772 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a92619c-92d4-4414-b801-861a2f44e2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:54.915325 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a92619c-92d4-4414-b801-861a2f44e2f9 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:54.916212 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a92619c-92d4-4414-b801-861a2f44e2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:54.916525 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a92619c-92d4-4414-b801-861a2f44e2f9 None admin] PATH_INFO: `/v3/users/b3a765997bc54d43991214bfaaf30437` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:54.917677 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a92619c-92d4-4414-b801-861a2f44e2f9 None admin] RBAC: Authorizing `identity:get_user(user_id=b3a765997bc54d43991214bfaaf30437)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:54.919110 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a92619c-92d4-4414-b801-861a2f44e2f9 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:54.920702 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 552/1127] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 8 01:01:54 2022] GET /identity/v3/users/b3a765997bc54d43991214bfaaf30437 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:54.926475 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc0981-38e3-4603-a593-870178357d77 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:54.943894 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:55.027819 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc0981-38e3-4603-a593-870178357d77 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:55.060822 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc43d9c-5075-4767-8607-a5452aa19de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:55.061402 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc43d9c-5075-4767-8607-a5452aa19de7 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:55.061674 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc43d9c-5075-4767-8607-a5452aa19de7 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:55.061943 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc43d9c-5075-4767-8607-a5452aa19de7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:55.127652 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc43d9c-5075-4767-8607-a5452aa19de7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:55.129123 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc43d9c-5075-4767-8607-a5452aa19de7 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:55.145665 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:55.224732 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 576/1128] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 01:01:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 01:01:55.231628 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a92619c-92d4-4414-b801-861a2f44e2f9 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:55.323561 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:55.329501 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a92619c-92d4-4414-b801-861a2f44e2f9 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:55.362833 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ab1c28-8ce5-4bb4-bb81-c9e583967b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:55.363446 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab1c28-8ce5-4bb4-bb81-c9e583967b76 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:55.363726 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab1c28-8ce5-4bb4-bb81-c9e583967b76 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:55.364001 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab1c28-8ce5-4bb4-bb81-c9e583967b76 None admin] PATH_INFO: `/v3/projects/b898a6e4e2da4c09893c6aa1db674bd4` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:55.365194 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ab1c28-8ce5-4bb4-bb81-c9e583967b76 None admin] RBAC: Authorizing `identity:get_project(project_id=b898a6e4e2da4c09893c6aa1db674bd4)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:55.366639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ab1c28-8ce5-4bb4-bb81-c9e583967b76 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:55.368278 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 553/1129] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 01:01:55 2022] GET /identity/v3/projects/b898a6e4e2da4c09893c6aa1db674bd4 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:55.374452 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc43d9c-5075-4767-8607-a5452aa19de7 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:55.470610 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc43d9c-5075-4767-8607-a5452aa19de7 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:55.486383 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:55.504045 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16380010-7ef1-4b50-bd52-22a0f792434b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:55.504657 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16380010-7ef1-4b50-bd52-22a0f792434b None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:55.504933 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16380010-7ef1-4b50-bd52-22a0f792434b None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:55.505204 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16380010-7ef1-4b50-bd52-22a0f792434b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:55.505919 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16380010-7ef1-4b50-bd52-22a0f792434b None admin] RBAC: Adding query filter params (role.id=069fe3e033664493b3def844e3735bc9, scope.project.id=b898a6e4e2da4c09893c6aa1db674bd4, user.id=b3a765997bc54d43991214bfaaf30437) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:01:55.506224 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16380010-7ef1-4b50-bd52-22a0f792434b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:55.507625 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16380010-7ef1-4b50-bd52-22a0f792434b None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:55.514003 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 577/1130] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 8 01:01:55 2022] GET /identity/v3/role_assignments?user.id=b3a765997bc54d43991214bfaaf30437&scope.project.id=b898a6e4e2da4c09893c6aa1db674bd4&role.id=069fe3e033664493b3def844e3735bc9 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:57.672964 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe2eba5-1da1-4f91-9b7f-2fae817f4c4e None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:57.673428 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe2eba5-1da1-4f91-9b7f-2fae817f4c4e None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:57.673519 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe2eba5-1da1-4f91-9b7f-2fae817f4c4e None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:57.674847 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 554/1131] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:01:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:01:57.680369 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a88c27d-479b-439e-8ba1-c1c1278db8ad None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:57.680788 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a88c27d-479b-439e-8ba1-c1c1278db8ad None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:57.680969 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a88c27d-479b-439e-8ba1-c1c1278db8ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:57.699112 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-0a88c27d-479b-439e-8ba1-c1c1278db8ad None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:01:57.700322 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-0a88c27d-479b-439e-8ba1-c1c1278db8ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:01:57.764263 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 578/1132] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:01:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 01:01:58.032144 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe2eba5-1da1-4f91-9b7f-2fae817f4c4e None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:58.131368 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe2eba5-1da1-4f91-9b7f-2fae817f4c4e None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:58.165969 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9414fe30-a5c2-458d-ad2a-b815955d1689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:58.166706 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414fe30-a5c2-458d-ad2a-b815955d1689 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:58.166989 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414fe30-a5c2-458d-ad2a-b815955d1689 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:58.167264 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414fe30-a5c2-458d-ad2a-b815955d1689 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:58.168403 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9414fe30-a5c2-458d-ad2a-b815955d1689 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:58.169840 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9414fe30-a5c2-458d-ad2a-b815955d1689 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:58.171390 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 555/1133] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:01:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:01:58.177393 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a88c27d-479b-439e-8ba1-c1c1278db8ad None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:01:58.273447 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a88c27d-479b-439e-8ba1-c1c1278db8ad None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:01:58.284367 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:01:58.307424 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c655891e-b8a9-420c-92d3-0032c581b123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:01:58.308016 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c655891e-b8a9-420c-92d3-0032c581b123 None admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:01:58.308322 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c655891e-b8a9-420c-92d3-0032c581b123 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:01:58.308595 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c655891e-b8a9-420c-92d3-0032c581b123 None admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:01:58.309235 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c655891e-b8a9-420c-92d3-0032c581b123 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:01:58.310660 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c655891e-b8a9-420c-92d3-0032c581b123 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:01:58.336420 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 579/1134] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Sep 8 01:01:58 2022] POST /identity/v3/projects => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:02:00.486426 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9546a6c9-bc04-44d0-8059-11c977cb129f None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:00.486991 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9546a6c9-bc04-44d0-8059-11c977cb129f None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:00.486991 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9546a6c9-bc04-44d0-8059-11c977cb129f None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:00.488367 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 556/1135] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:02:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:02:00.493953 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb120e3-c9ff-40a5-a90a-430d9395f81c None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:00.494384 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb120e3-c9ff-40a5-a90a-430d9395f81c None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:00.494604 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb120e3-c9ff-40a5-a90a-430d9395f81c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:00.513133 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-4bb120e3-c9ff-40a5-a90a-430d9395f81c None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:02:00.514304 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-4bb120e3-c9ff-40a5-a90a-430d9395f81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:02:00.579283 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 580/1136] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:02:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 01:02:00.847047 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9546a6c9-bc04-44d0-8059-11c977cb129f None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:02:00.878019 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:02:00.947543 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9546a6c9-bc04-44d0-8059-11c977cb129f None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:02:00.980990 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8064a2de-6e3d-445e-a200-075cfd08b3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:02:00.981610 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8064a2de-6e3d-445e-a200-075cfd08b3c3 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:00.981892 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8064a2de-6e3d-445e-a200-075cfd08b3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:00.982167 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8064a2de-6e3d-445e-a200-075cfd08b3c3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:00.983269 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8064a2de-6e3d-445e-a200-075cfd08b3c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:02:00.984751 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8064a2de-6e3d-445e-a200-075cfd08b3c3 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:02:00.986314 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 557/1137] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:02:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:02:00.991986 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb120e3-c9ff-40a5-a90a-430d9395f81c None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:02:01.078236 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:02:01.088700 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb120e3-c9ff-40a5-a90a-430d9395f81c None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:02:01.121374 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fbb402-5821-4ed0-886a-fdc944e4018c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:02:01.121993 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fbb402-5821-4ed0-886a-fdc944e4018c None admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:01.122271 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fbb402-5821-4ed0-886a-fdc944e4018c None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:01.122538 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fbb402-5821-4ed0-886a-fdc944e4018c None admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:01.123182 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fbb402-5821-4ed0-886a-fdc944e4018c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:02:01.124629 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fbb402-5821-4ed0-886a-fdc944e4018c None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:02:01.145407 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 581/1138] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 8 01:02:00 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:02:03.328748 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5520f009-7ce7-4263-8d57-c1cb31f31c8e None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:03.329308 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5520f009-7ce7-4263-8d57-c1cb31f31c8e None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:03.329308 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5520f009-7ce7-4263-8d57-c1cb31f31c8e None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:03.330611 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 558/1139] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:02:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:02:03.335924 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f4c697-dcb5-4a13-8331-4f4e58c5ae26 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:03.336381 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f4c697-dcb5-4a13-8331-4f4e58c5ae26 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:03.336499 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f4c697-dcb5-4a13-8331-4f4e58c5ae26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:03.355431 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-21f4c697-dcb5-4a13-8331-4f4e58c5ae26 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:02:03.356712 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-21f4c697-dcb5-4a13-8331-4f4e58c5ae26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:02:03.421569 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 582/1140] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:02:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 01:02:03.692711 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5520f009-7ce7-4263-8d57-c1cb31f31c8e None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:02:03.787958 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:02:03.792347 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5520f009-7ce7-4263-8d57-c1cb31f31c8e None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:02:03.825291 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aa3f3b-bf9d-4331-aa03-484b230680fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:02:03.825899 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa3f3b-bf9d-4331-aa03-484b230680fe None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:03.826176 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa3f3b-bf9d-4331-aa03-484b230680fe None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:03.826441 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa3f3b-bf9d-4331-aa03-484b230680fe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:03.836026 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aa3f3b-bf9d-4331-aa03-484b230680fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:02:03.841667 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-78aa3f3b-bf9d-4331-aa03-484b230680fe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 01:02:03.841667 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:02:03.841667 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:02:03.841667 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:02:03.841667 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:02:03.841667 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:02:03.841667 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:02:03.841667 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:02:03.841667 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:02:03.841667 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:02:03.841667 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:02:03.841667 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:02:03.841667 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:02:03.841667 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:02:03.841667 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:02:03.841667 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:02:03.842777 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:02:03.842777 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:02:03.842777 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:02:03.842777 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:02:03.842777 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:02:03.842777 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:02:03.842777 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:02:03.842777 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:02:03.842777 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:02:03.842777 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:02:03.842777 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:02:03.842777 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:02:03.842777 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:02:03.842777 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:02:03.842777 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:02:03.842777 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:02:03.843780 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:02:03.843780 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:02:03.843780 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:02:03.843780 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:02:03.843780 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:02:03.843780 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:02:03.843780 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:02:03.843780 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:02:03.843780 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:02:03.843780 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:02:03.843780 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 01:02:03.843780 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:02:03.843780 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 559/1141] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 8 01:02:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:02:03.848839 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f4c697-dcb5-4a13-8331-4f4e58c5ae26 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:02:03.923704 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:02:03.945520 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f4c697-dcb5-4a13-8331-4f4e58c5ae26 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:02:03.979311 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f3f051-7ddb-4a94-8c67-973b0833124a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:02:03.979898 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f3f051-7ddb-4a94-8c67-973b0833124a None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:03.980193 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f3f051-7ddb-4a94-8c67-973b0833124a None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:03.980463 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f3f051-7ddb-4a94-8c67-973b0833124a None admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:03.981115 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f3f051-7ddb-4a94-8c67-973b0833124a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:02:03.981412 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f3f051-7ddb-4a94-8c67-973b0833124a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:02:03.982759 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f3f051-7ddb-4a94-8c67-973b0833124a None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:02:03.990396 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 583/1142] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 01:02:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:02:03.996840 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aa3f3b-bf9d-4331-aa03-484b230680fe None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:02:04.093097 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aa3f3b-bf9d-4331-aa03-484b230680fe None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:02:04.126313 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f76af3d-2b5e-4e50-800c-306b959c7e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:02:04.126917 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76af3d-2b5e-4e50-800c-306b959c7e74 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:04.127192 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76af3d-2b5e-4e50-800c-306b959c7e74 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:04.127459 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76af3d-2b5e-4e50-800c-306b959c7e74 None admin] PATH_INFO: `/v3/users/1c55ea4efa1d4567b6ad119e83595af4` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:04.140282 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f76af3d-2b5e-4e50-800c-306b959c7e74 None admin] RBAC: Authorizing `identity:get_user(user_id=1c55ea4efa1d4567b6ad119e83595af4)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:02:04.141700 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f76af3d-2b5e-4e50-800c-306b959c7e74 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:02:04.143240 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 560/1143] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 8 01:02:03 2022] GET /identity/v3/users/1c55ea4efa1d4567b6ad119e83595af4 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:02:04.149103 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f3f051-7ddb-4a94-8c67-973b0833124a None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:02:04.246629 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f3f051-7ddb-4a94-8c67-973b0833124a None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:02:04.279342 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b46e5-d682-4471-b96b-54d1a80086d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:02:04.279930 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b46e5-d682-4471-b96b-54d1a80086d6 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:04.280234 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b46e5-d682-4471-b96b-54d1a80086d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:04.280512 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b46e5-d682-4471-b96b-54d1a80086d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:04.290979 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:02:04.346123 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85b46e5-d682-4471-b96b-54d1a80086d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:02:04.347555 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85b46e5-d682-4471-b96b-54d1a80086d6 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:02:04.442443 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 584/1144] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 01:02:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 01:02:04.448758 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f76af3d-2b5e-4e50-800c-306b959c7e74 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:02:04.466886 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:02:04.544697 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f76af3d-2b5e-4e50-800c-306b959c7e74 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:02:04.577356 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d5fbe-49a1-402d-9865-fad8000be9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:02:04.577964 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d5fbe-49a1-402d-9865-fad8000be9d3 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:04.578241 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d5fbe-49a1-402d-9865-fad8000be9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:04.578510 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d5fbe-49a1-402d-9865-fad8000be9d3 None admin] PATH_INFO: `/v3/projects/38b5d7c601bb4d8abc4aebf40eff9614` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:04.579605 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d5fbe-49a1-402d-9865-fad8000be9d3 None admin] RBAC: Authorizing `identity:get_project(project_id=38b5d7c601bb4d8abc4aebf40eff9614)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:02:04.581052 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d5fbe-49a1-402d-9865-fad8000be9d3 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:02:04.582631 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 561/1145] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 01:02:04 2022] GET /identity/v3/projects/38b5d7c601bb4d8abc4aebf40eff9614 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:02:04.589090 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b46e5-d682-4471-b96b-54d1a80086d6 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:02:04.630247 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:02:04.687321 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b46e5-d682-4471-b96b-54d1a80086d6 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:02:04.721223 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9051132f-c2b0-4746-9fe6-58fb40c827de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:02:04.721815 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9051132f-c2b0-4746-9fe6-58fb40c827de None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:04.722092 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9051132f-c2b0-4746-9fe6-58fb40c827de None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:04.722361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9051132f-c2b0-4746-9fe6-58fb40c827de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:04.723076 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9051132f-c2b0-4746-9fe6-58fb40c827de None admin] RBAC: Adding query filter params (role.id=34de369ec7c54715aa83a9052a26c3f2, scope.project.id=38b5d7c601bb4d8abc4aebf40eff9614, user.id=1c55ea4efa1d4567b6ad119e83595af4) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:02:04.723379 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9051132f-c2b0-4746-9fe6-58fb40c827de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:02:04.724822 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9051132f-c2b0-4746-9fe6-58fb40c827de None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:02:04.730653 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 585/1146] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 8 01:02:04 2022] GET /identity/v3/role_assignments?user.id=1c55ea4efa1d4567b6ad119e83595af4&scope.project.id=38b5d7c601bb4d8abc4aebf40eff9614&role.id=34de369ec7c54715aa83a9052a26c3f2 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:02:06.841156 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c763f-9dee-4bcc-94d7-1198187c712f None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:06.841662 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c763f-9dee-4bcc-94d7-1198187c712f None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:06.841781 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c763f-9dee-4bcc-94d7-1198187c712f None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:06.843012 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 562/1147] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:02:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:02:06.848228 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0217f95-6603-4912-b751-a84d632d97a8 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:06.848654 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0217f95-6603-4912-b751-a84d632d97a8 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:06.848776 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0217f95-6603-4912-b751-a84d632d97a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:06.866959 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-c0217f95-6603-4912-b751-a84d632d97a8 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:02:06.868205 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-c0217f95-6603-4912-b751-a84d632d97a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:02:06.933954 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 586/1148] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:02:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 01:02:07.205654 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c763f-9dee-4bcc-94d7-1198187c712f None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:02:07.290877 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:02:07.307120 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c763f-9dee-4bcc-94d7-1198187c712f None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:02:07.340139 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae9a6e-4d97-45ce-a37f-141fa7450b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:02:07.340777 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae9a6e-4d97-45ce-a37f-141fa7450b55 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:07.341053 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae9a6e-4d97-45ce-a37f-141fa7450b55 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:07.341320 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae9a6e-4d97-45ce-a37f-141fa7450b55 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:07.351074 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcae9a6e-4d97-45ce-a37f-141fa7450b55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:02:07.356375 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-fcae9a6e-4d97-45ce-a37f-141fa7450b55 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 01:02:07.356375 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:02:07.356375 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:02:07.356375 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:02:07.356375 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:02:07.356375 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:02:07.356375 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:02:07.356375 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:02:07.356375 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:02:07.356375 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:02:07.356375 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:02:07.356375 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:02:07.356375 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:02:07.356375 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:02:07.356375 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:02:07.356375 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:02:07.357456 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:02:07.357456 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:02:07.357456 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:02:07.357456 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:02:07.357456 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:02:07.357456 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:02:07.357456 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:02:07.357456 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:02:07.357456 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:02:07.357456 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:02:07.357456 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:02:07.357456 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:02:07.357456 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:02:07.357456 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:02:07.357456 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:02:07.357456 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:02:07.358557 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:02:07.358557 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:02:07.358557 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:02:07.358557 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:02:07.358557 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:02:07.358557 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:02:07.358557 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:02:07.358557 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:02:07.358557 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:02:07.358557 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:02:07.358557 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 01:02:07.358557 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:02:07.358557 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 563/1149] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 8 01:02:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:02:07.363495 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0217f95-6603-4912-b751-a84d632d97a8 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:02:07.396366 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:02:07.461302 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0217f95-6603-4912-b751-a84d632d97a8 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:02:07.494564 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3928ce3-743d-45dc-844d-8dd3da48f8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:02:07.495153 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3928ce3-743d-45dc-844d-8dd3da48f8f5 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:07.495431 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3928ce3-743d-45dc-844d-8dd3da48f8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:07.495701 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3928ce3-743d-45dc-844d-8dd3da48f8f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:07.496380 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3928ce3-743d-45dc-844d-8dd3da48f8f5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:02:07.496682 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3928ce3-743d-45dc-844d-8dd3da48f8f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:02:07.498042 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3928ce3-743d-45dc-844d-8dd3da48f8f5 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:02:07.505786 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 587/1150] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 01:02:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:02:07.511790 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae9a6e-4d97-45ce-a37f-141fa7450b55 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:02:07.608026 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae9a6e-4d97-45ce-a37f-141fa7450b55 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:02:07.641300 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072f75dc-fd24-498e-b081-4bbf3b74fc91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:02:07.641920 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f75dc-fd24-498e-b081-4bbf3b74fc91 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:07.642199 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f75dc-fd24-498e-b081-4bbf3b74fc91 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:07.642471 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f75dc-fd24-498e-b081-4bbf3b74fc91 None admin] PATH_INFO: `/v3/users/1c55ea4efa1d4567b6ad119e83595af4` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:07.643569 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072f75dc-fd24-498e-b081-4bbf3b74fc91 None admin] RBAC: Authorizing `identity:get_user(user_id=1c55ea4efa1d4567b6ad119e83595af4)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:02:07.645039 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072f75dc-fd24-498e-b081-4bbf3b74fc91 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:02:07.646599 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 564/1151] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 8 01:02:07 2022] GET /identity/v3/users/1c55ea4efa1d4567b6ad119e83595af4 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:02:07.653140 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3928ce3-743d-45dc-844d-8dd3da48f8f5 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:02:07.744635 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:02:07.750962 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3928ce3-743d-45dc-844d-8dd3da48f8f5 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:02:07.783672 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5c9ec8-544c-4e6e-8eb9-387893048e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:02:07.784328 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c9ec8-544c-4e6e-8eb9-387893048e76 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:07.784610 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c9ec8-544c-4e6e-8eb9-387893048e76 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:07.784885 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c9ec8-544c-4e6e-8eb9-387893048e76 None admin] PATH_INFO: `/v3/projects/38b5d7c601bb4d8abc4aebf40eff9614` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:07.785991 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5c9ec8-544c-4e6e-8eb9-387893048e76 None admin] RBAC: Authorizing `identity:get_project(project_id=38b5d7c601bb4d8abc4aebf40eff9614)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:02:07.787420 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5c9ec8-544c-4e6e-8eb9-387893048e76 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:02:07.789053 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 588/1152] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 01:02:07 2022] GET /identity/v3/projects/38b5d7c601bb4d8abc4aebf40eff9614 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:02:07.795516 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072f75dc-fd24-498e-b081-4bbf3b74fc91 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:02:07.811216 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:02:07.891861 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072f75dc-fd24-498e-b081-4bbf3b74fc91 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:02:07.923925 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce968e9-f9fd-4319-aadf-5820c262d46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:02:07.924587 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce968e9-f9fd-4319-aadf-5820c262d46a None admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:07.924864 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce968e9-f9fd-4319-aadf-5820c262d46a None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:07.925135 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce968e9-f9fd-4319-aadf-5820c262d46a None admin] PATH_INFO: `/v3/projects/38b5d7c601bb4d8abc4aebf40eff9614/users/1c55ea4efa1d4567b6ad119e83595af4/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:07.926981 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce968e9-f9fd-4319-aadf-5820c262d46a None admin] RBAC: Authorizing `identity:create_grant(project_id=38b5d7c601bb4d8abc4aebf40eff9614, user_id=1c55ea4efa1d4567b6ad119e83595af4, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:02:07.928446 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce968e9-f9fd-4319-aadf-5820c262d46a None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:02:07.941481 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 565/1153] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 8 01:02:07 2022] PUT /identity/v3/projects/38b5d7c601bb4d8abc4aebf40eff9614/users/1c55ea4efa1d4567b6ad119e83595af4/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:02:10.121238 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaeb988-362d-45fc-be09-ccdb1307ba23 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:10.121816 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaeb988-362d-45fc-be09-ccdb1307ba23 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:10.121816 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaeb988-362d-45fc-be09-ccdb1307ba23 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:10.123119 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 589/1154] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:02:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:02:10.128769 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44976555-aa8d-4b6c-a6ac-d94a529d38e8 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:10.129488 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44976555-aa8d-4b6c-a6ac-d94a529d38e8 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:10.129488 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44976555-aa8d-4b6c-a6ac-d94a529d38e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:10.148071 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-44976555-aa8d-4b6c-a6ac-d94a529d38e8 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:02:10.149320 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-44976555-aa8d-4b6c-a6ac-d94a529d38e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:02:10.215679 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 566/1155] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:02:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 01:02:10.505299 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaeb988-362d-45fc-be09-ccdb1307ba23 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:02:10.604604 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaeb988-362d-45fc-be09-ccdb1307ba23 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:02:10.637711 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce137cb-a0c5-40c6-8858-a612846dd4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:02:10.638332 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce137cb-a0c5-40c6-8858-a612846dd4b7 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:10.638610 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce137cb-a0c5-40c6-8858-a612846dd4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:10.638878 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce137cb-a0c5-40c6-8858-a612846dd4b7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:10.649106 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce137cb-a0c5-40c6-8858-a612846dd4b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:02:10.655388 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-7ce137cb-a0c5-40c6-8858-a612846dd4b7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 01:02:10.655388 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:02:10.655388 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:02:10.655388 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:02:10.655388 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:02:10.655388 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:02:10.655388 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:02:10.655388 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:02:10.655388 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:02:10.655388 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:02:10.655388 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:02:10.655388 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:02:10.655388 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:02:10.655388 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:02:10.655388 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:02:10.655388 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:02:10.656605 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:02:10.656605 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:02:10.656605 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:02:10.656605 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:02:10.656605 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:02:10.656605 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:02:10.656605 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:02:10.656605 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:02:10.656605 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:02:10.656605 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:02:10.656605 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:02:10.656605 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:02:10.656605 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:02:10.656605 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:02:10.656605 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:02:10.656605 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:02:10.657815 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:02:10.657815 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:02:10.657815 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:02:10.657815 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:02:10.657815 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:02:10.657815 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:02:10.657815 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:02:10.657815 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:02:10.657815 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:02:10.657815 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:02:10.657815 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 01:02:10.657815 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:02:10.657815 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 590/1156] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 8 01:02:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 01:02:10.662671 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44976555-aa8d-4b6c-a6ac-d94a529d38e8 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:02:10.666380 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:02:10.758602 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44976555-aa8d-4b6c-a6ac-d94a529d38e8 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:02:10.792151 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be067400-8319-41fc-bebc-8159574ff95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:02:10.792740 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be067400-8319-41fc-bebc-8159574ff95b None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:10.793013 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be067400-8319-41fc-bebc-8159574ff95b None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:10.793279 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be067400-8319-41fc-bebc-8159574ff95b None admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:10.793924 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be067400-8319-41fc-bebc-8159574ff95b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:02:10.794220 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be067400-8319-41fc-bebc-8159574ff95b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:02:10.795574 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be067400-8319-41fc-bebc-8159574ff95b None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:02:10.803173 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 567/1157] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 01:02:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:02:10.809246 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce137cb-a0c5-40c6-8858-a612846dd4b7 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:02:10.833783 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:02:10.905824 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce137cb-a0c5-40c6-8858-a612846dd4b7 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:02:10.937861 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabb590f-67bc-49e9-abab-1bcd6daca835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:02:10.938469 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabb590f-67bc-49e9-abab-1bcd6daca835 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:10.938744 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabb590f-67bc-49e9-abab-1bcd6daca835 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:10.939010 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabb590f-67bc-49e9-abab-1bcd6daca835 None admin] PATH_INFO: `/v3/users/1c55ea4efa1d4567b6ad119e83595af4` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:10.940097 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabb590f-67bc-49e9-abab-1bcd6daca835 None admin] RBAC: Authorizing `identity:get_user(user_id=1c55ea4efa1d4567b6ad119e83595af4)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:02:10.941543 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabb590f-67bc-49e9-abab-1bcd6daca835 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:02:10.943080 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 591/1158] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 8 01:02:10 2022] GET /identity/v3/users/1c55ea4efa1d4567b6ad119e83595af4 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:02:10.948923 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be067400-8319-41fc-bebc-8159574ff95b None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:02:11.013691 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:02:11.048174 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be067400-8319-41fc-bebc-8159574ff95b None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:02:11.080691 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb287436-4383-491a-9a00-b6b19696e41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:02:11.081288 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb287436-4383-491a-9a00-b6b19696e41d None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:11.081567 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb287436-4383-491a-9a00-b6b19696e41d None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:11.081841 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb287436-4383-491a-9a00-b6b19696e41d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:11.146102 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb287436-4383-491a-9a00-b6b19696e41d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:02:11.147562 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb287436-4383-491a-9a00-b6b19696e41d None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:02:11.209927 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:02:11.244797 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 568/1159] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 01:02:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 01:02:11.250776 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabb590f-67bc-49e9-abab-1bcd6daca835 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:02:11.341080 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:02:11.349914 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabb590f-67bc-49e9-abab-1bcd6daca835 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:02:11.384240 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5945bc75-e9f4-4952-a2bc-56b67ff081eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:02:11.384854 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945bc75-e9f4-4952-a2bc-56b67ff081eb None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:11.385131 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945bc75-e9f4-4952-a2bc-56b67ff081eb None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:11.385402 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945bc75-e9f4-4952-a2bc-56b67ff081eb None admin] PATH_INFO: `/v3/projects/38b5d7c601bb4d8abc4aebf40eff9614` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:11.386759 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5945bc75-e9f4-4952-a2bc-56b67ff081eb None admin] RBAC: Authorizing `identity:get_project(project_id=38b5d7c601bb4d8abc4aebf40eff9614)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:02:11.388208 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5945bc75-e9f4-4952-a2bc-56b67ff081eb None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:02:11.389819 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 592/1160] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 01:02:11 2022] GET /identity/v3/projects/38b5d7c601bb4d8abc4aebf40eff9614 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:02:11.395959 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb287436-4383-491a-9a00-b6b19696e41d None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:02:11.492623 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb287436-4383-491a-9a00-b6b19696e41d None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:02:11.526015 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe33576-0ba6-4ba4-ab96-dc29de47bfb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:02:11.526607 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe33576-0ba6-4ba4-ab96-dc29de47bfb4 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:11.526888 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe33576-0ba6-4ba4-ab96-dc29de47bfb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:11.527159 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe33576-0ba6-4ba4-ab96-dc29de47bfb4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:11.527886 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe33576-0ba6-4ba4-ab96-dc29de47bfb4 None admin] RBAC: Adding query filter params (role.id=34de369ec7c54715aa83a9052a26c3f2, scope.project.id=38b5d7c601bb4d8abc4aebf40eff9614, user.id=1c55ea4efa1d4567b6ad119e83595af4) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:02:11.528218 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe33576-0ba6-4ba4-ab96-dc29de47bfb4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:02:11.529646 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe33576-0ba6-4ba4-ab96-dc29de47bfb4 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:02:11.535198 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 569/1161] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 8 01:02:11 2022] GET /identity/v3/role_assignments?user.id=1c55ea4efa1d4567b6ad119e83595af4&scope.project.id=38b5d7c601bb4d8abc4aebf40eff9614&role.id=34de369ec7c54715aa83a9052a26c3f2 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:02:13.714686 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20367e5c-1b29-4427-b49d-27eeeeea6a40 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:13.715201 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20367e5c-1b29-4427-b49d-27eeeeea6a40 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:13.715201 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20367e5c-1b29-4427-b49d-27eeeeea6a40 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:13.716574 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 593/1162] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:02:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:02:13.721609 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d81fed-2497-495a-8453-eb5db2b5875f None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:13.722039 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d81fed-2497-495a-8453-eb5db2b5875f None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:13.722137 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d81fed-2497-495a-8453-eb5db2b5875f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:13.740057 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-41d81fed-2497-495a-8453-eb5db2b5875f None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:02:13.741259 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-41d81fed-2497-495a-8453-eb5db2b5875f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:02:13.765239 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:02:13.807655 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 570/1163] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:02:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 01:02:14.072567 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20367e5c-1b29-4427-b49d-27eeeeea6a40 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:02:14.171114 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20367e5c-1b29-4427-b49d-27eeeeea6a40 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:02:14.203691 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bff8795-b5ed-4152-a4bd-79cf1ad156b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:02:14.204307 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bff8795-b5ed-4152-a4bd-79cf1ad156b0 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:14.204581 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bff8795-b5ed-4152-a4bd-79cf1ad156b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:14.204848 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bff8795-b5ed-4152-a4bd-79cf1ad156b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:14.231337 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:02:14.270930 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bff8795-b5ed-4152-a4bd-79cf1ad156b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:02:14.272381 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bff8795-b5ed-4152-a4bd-79cf1ad156b0 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:02:14.365751 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 594/1164] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 01:02:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 01:02:14.372318 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d81fed-2497-495a-8453-eb5db2b5875f None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:02:14.472267 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d81fed-2497-495a-8453-eb5db2b5875f None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:02:14.507367 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2baa4a-cbee-4b5d-a86d-f4559c9b203c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:02:14.507996 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2baa4a-cbee-4b5d-a86d-f4559c9b203c None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:14.508306 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2baa4a-cbee-4b5d-a86d-f4559c9b203c None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:14.508715 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2baa4a-cbee-4b5d-a86d-f4559c9b203c None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:14.523148 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2baa4a-cbee-4b5d-a86d-f4559c9b203c None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:02:14.525244 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2baa4a-cbee-4b5d-a86d-f4559c9b203c None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:02:14.543143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-9a2baa4a-cbee-4b5d-a86d-f4559c9b203c None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 08 01:02:14.543143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:02:14.543143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 08 01:02:14.543143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 08 01:02:14.543143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 01:02:14.543143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 01:02:14.543143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 01:02:14.543143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 01:02:14.543143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 08 01:02:14.543143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:02:14.543143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 01:02:14.543143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:02:14.543143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:02:14.543143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:02:14.543143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:02:14.543143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:02:14.543143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:02:14.544559 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:02:14.544559 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:02:14.544559 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:02:14.544559 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:02:14.544559 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:02:14.544559 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:02:14.544559 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:02:14.544559 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:02:14.544559 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:02:14.544559 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:02:14.544559 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:02:14.544559 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:02:14.544559 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:02:14.544559 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:02:14.544559 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:02:14.544559 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:02:14.545620 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:02:14.545620 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:02:14.545620 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:02:14.545620 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:02:14.545620 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:02:14.545620 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:02:14.545620 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:02:14.545620 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:02:14.545620 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:02:14.545620 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:02:14.545620 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 08 01:02:14.545620 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:02:14.545620 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 08 01:02:14.545620 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:02:14.548293 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 571/1165] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Sep 8 01:02:14 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:02:14.552239 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bff8795-b5ed-4152-a4bd-79cf1ad156b0 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:02:14.614493 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:02:14.661541 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bff8795-b5ed-4152-a4bd-79cf1ad156b0 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:02:14.697959 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ec0b3e-fb5f-44e7-ac1d-ceef73c420ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:02:14.698892 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec0b3e-fb5f-44e7-ac1d-ceef73c420ac None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:14.699218 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec0b3e-fb5f-44e7-ac1d-ceef73c420ac None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:14.699491 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec0b3e-fb5f-44e7-ac1d-ceef73c420ac None admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:14.700495 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ec0b3e-fb5f-44e7-ac1d-ceef73c420ac None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:02:14.700804 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ec0b3e-fb5f-44e7-ac1d-ceef73c420ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:02:14.702175 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ec0b3e-fb5f-44e7-ac1d-ceef73c420ac None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:02:14.711341 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 595/1166] 10.222.0.42 () {56 vars in 1115 bytes} [Thu Sep 8 01:02:14 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:02:16.900432 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d5f1f-55a2-484d-b620-df575bb50010 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:16.900944 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d5f1f-55a2-484d-b620-df575bb50010 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:16.900944 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d5f1f-55a2-484d-b620-df575bb50010 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:16.902234 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 572/1167] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:02:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:02:16.907515 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b84100f-b0ba-4cc4-b549-797ecd9e5d60 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:16.907952 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b84100f-b0ba-4cc4-b549-797ecd9e5d60 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:16.908085 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b84100f-b0ba-4cc4-b549-797ecd9e5d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:16.926669 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-9b84100f-b0ba-4cc4-b549-797ecd9e5d60 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:02:16.927866 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-9b84100f-b0ba-4cc4-b549-797ecd9e5d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:02:16.991638 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 596/1168] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:02:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 01:02:17.256450 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d5f1f-55a2-484d-b620-df575bb50010 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:02:17.279343 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:02:17.357195 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d5f1f-55a2-484d-b620-df575bb50010 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:02:17.390098 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf31144a-c05c-494e-9432-e46c8444b827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:02:17.390689 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31144a-c05c-494e-9432-e46c8444b827 None admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:17.390970 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31144a-c05c-494e-9432-e46c8444b827 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:17.391242 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31144a-c05c-494e-9432-e46c8444b827 None admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:17.391895 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf31144a-c05c-494e-9432-e46c8444b827 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:02:17.393303 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf31144a-c05c-494e-9432-e46c8444b827 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:02:17.408717 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 573/1169] 10.222.0.42 () {60 vars in 1138 bytes} [Thu Sep 8 01:02:17 2022] POST /identity/v3/domains => generated 267 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:02:19.579441 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ff1ab-e8b0-4657-bb3f-ad5fb21c64b1 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:19.579994 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ff1ab-e8b0-4657-bb3f-ad5fb21c64b1 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:19.579994 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ff1ab-e8b0-4657-bb3f-ad5fb21c64b1 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:19.581375 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 597/1170] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:02:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:02:19.586436 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb7a6dd-6346-4e60-9cf7-3da348271cb0 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:19.587509 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb7a6dd-6346-4e60-9cf7-3da348271cb0 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:19.587966 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb7a6dd-6346-4e60-9cf7-3da348271cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:19.605758 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-1fb7a6dd-6346-4e60-9cf7-3da348271cb0 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:02:19.606931 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-1fb7a6dd-6346-4e60-9cf7-3da348271cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:02:19.673308 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 574/1171] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:02:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 01:02:19.939944 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77ff1ab-e8b0-4657-bb3f-ad5fb21c64b1 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:02:19.994256 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:02:20.038048 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77ff1ab-e8b0-4657-bb3f-ad5fb21c64b1 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:02:20.071809 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30afd96d-e93c-48be-8017-c3fb4eaa0457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:02:20.072431 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30afd96d-e93c-48be-8017-c3fb4eaa0457 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:20.072704 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30afd96d-e93c-48be-8017-c3fb4eaa0457 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:20.072968 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30afd96d-e93c-48be-8017-c3fb4eaa0457 None admin] PATH_INFO: `/v3/domains/6e85e54b0f1d4187a2c7048a0378114f` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:20.082898 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30afd96d-e93c-48be-8017-c3fb4eaa0457 None admin] RBAC: Authorizing `identity:get_domain(domain_id=6e85e54b0f1d4187a2c7048a0378114f)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:02:20.084345 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30afd96d-e93c-48be-8017-c3fb4eaa0457 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:02:20.085880 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 598/1172] 10.222.0.42 () {56 vars in 1183 bytes} [Thu Sep 8 01:02:19 2022] GET /identity/v3/domains/6e85e54b0f1d4187a2c7048a0378114f => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:02:20.092525 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb7a6dd-6346-4e60-9cf7-3da348271cb0 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:02:20.103835 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:02:20.191243 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb7a6dd-6346-4e60-9cf7-3da348271cb0 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:02:20.224680 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea535b5f-9266-45ee-8e76-4ddc88ae2367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:02:20.225283 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea535b5f-9266-45ee-8e76-4ddc88ae2367 None admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:20.225563 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea535b5f-9266-45ee-8e76-4ddc88ae2367 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:20.225837 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea535b5f-9266-45ee-8e76-4ddc88ae2367 None admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:20.226491 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea535b5f-9266-45ee-8e76-4ddc88ae2367 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:02:20.227980 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea535b5f-9266-45ee-8e76-4ddc88ae2367 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:02:20.258943 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 575/1173] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Sep 8 01:02:20 2022] POST /identity/v3/projects => generated 361 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:02:22.424620 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4067d8-9437-46ac-8516-62ee1a38ae5b None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:22.425183 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4067d8-9437-46ac-8516-62ee1a38ae5b None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:22.425183 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4067d8-9437-46ac-8516-62ee1a38ae5b None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:22.426491 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 599/1174] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:02:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:02:22.432372 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194fcc0-695d-4a33-ac44-b57cc9493d54 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:22.432769 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194fcc0-695d-4a33-ac44-b57cc9493d54 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:22.432926 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194fcc0-695d-4a33-ac44-b57cc9493d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:22.450812 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-0194fcc0-695d-4a33-ac44-b57cc9493d54 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:02:22.451997 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-0194fcc0-695d-4a33-ac44-b57cc9493d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:02:22.506657 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:02:22.518124 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 576/1175] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:02:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 01:02:22.792890 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4067d8-9437-46ac-8516-62ee1a38ae5b None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:02:22.891545 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4067d8-9437-46ac-8516-62ee1a38ae5b None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:02:22.905228 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:02:22.925602 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee2de60-aca2-45f0-8200-acb67c45e56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:02:22.926217 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2de60-aca2-45f0-8200-acb67c45e56a None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:22.926496 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2de60-aca2-45f0-8200-acb67c45e56a None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:22.926767 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2de60-aca2-45f0-8200-acb67c45e56a None admin] PATH_INFO: `/v3/domains/6e85e54b0f1d4187a2c7048a0378114f` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:22.927867 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee2de60-aca2-45f0-8200-acb67c45e56a None admin] RBAC: Authorizing `identity:get_domain(domain_id=6e85e54b0f1d4187a2c7048a0378114f)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:02:22.929329 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee2de60-aca2-45f0-8200-acb67c45e56a None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:02:22.930955 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 600/1176] 10.222.0.42 () {56 vars in 1183 bytes} [Thu Sep 8 01:02:22 2022] GET /identity/v3/domains/6e85e54b0f1d4187a2c7048a0378114f => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:02:22.936767 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0194fcc0-695d-4a33-ac44-b57cc9493d54 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:02:23.033819 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0194fcc0-695d-4a33-ac44-b57cc9493d54 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:02:23.066055 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc8599b-8aed-4286-a1a0-dea698c49872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:02:23.066594 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc8599b-8aed-4286-a1a0-dea698c49872 None admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:23.066874 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc8599b-8aed-4286-a1a0-dea698c49872 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:23.067149 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc8599b-8aed-4286-a1a0-dea698c49872 None admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:23.067790 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc8599b-8aed-4286-a1a0-dea698c49872 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:02:23.069262 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc8599b-8aed-4286-a1a0-dea698c49872 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:02:23.090515 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 577/1177] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 8 01:02:22 2022] POST /identity/v3/users => generated 311 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:02:25.272754 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ed1ac-c257-47b7-ae0a-034a38dff79b None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:25.273283 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ed1ac-c257-47b7-ae0a-034a38dff79b None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:25.273283 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ed1ac-c257-47b7-ae0a-034a38dff79b None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:25.275108 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 601/1178] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:02:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:02:25.280467 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc29a878-a28d-4709-9c00-7114b722c08d None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:25.280841 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc29a878-a28d-4709-9c00-7114b722c08d None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:25.281021 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc29a878-a28d-4709-9c00-7114b722c08d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:25.299747 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-dc29a878-a28d-4709-9c00-7114b722c08d None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:02:25.300949 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-dc29a878-a28d-4709-9c00-7114b722c08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:02:25.344329 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:02:25.367435 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 578/1179] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:02:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 01:02:25.642541 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ed1ac-c257-47b7-ae0a-034a38dff79b None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:02:25.741577 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ed1ac-c257-47b7-ae0a-034a38dff79b None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:02:25.774445 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e99474-fe7b-4774-916d-a5f6a292ed76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:02:25.775314 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e99474-fe7b-4774-916d-a5f6a292ed76 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:25.775589 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e99474-fe7b-4774-916d-a5f6a292ed76 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:25.775856 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e99474-fe7b-4774-916d-a5f6a292ed76 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:25.785884 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e99474-fe7b-4774-916d-a5f6a292ed76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:02:25.791438 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-71e99474-fe7b-4774-916d-a5f6a292ed76 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 01:02:25.791438 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:02:25.791438 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:02:25.791438 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:02:25.791438 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:02:25.791438 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:02:25.791438 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:02:25.791438 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:02:25.791438 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:02:25.791438 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:02:25.791438 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:02:25.791438 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:02:25.791438 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:02:25.791438 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:02:25.791438 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:02:25.791438 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:02:25.792657 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:02:25.792657 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:02:25.792657 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:02:25.792657 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:02:25.792657 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:02:25.792657 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:02:25.792657 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:02:25.792657 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:02:25.792657 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:02:25.792657 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:02:25.792657 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:02:25.792657 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:02:25.792657 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:02:25.792657 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:02:25.792657 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:02:25.792657 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:02:25.793696 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:02:25.793696 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:02:25.793696 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:02:25.793696 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:02:25.793696 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:02:25.793696 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:02:25.793696 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:02:25.793696 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:02:25.793696 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:02:25.793696 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:02:25.793696 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 01:02:25.793696 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:02:25.793696 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 602/1180] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 8 01:02:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:02:25.798348 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc29a878-a28d-4709-9c00-7114b722c08d None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:02:25.895846 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc29a878-a28d-4709-9c00-7114b722c08d None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:02:25.929128 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c258947b-66d8-463c-b106-bfde5128ca5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:02:25.929717 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258947b-66d8-463c-b106-bfde5128ca5a None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:25.930072 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258947b-66d8-463c-b106-bfde5128ca5a None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:25.930348 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258947b-66d8-463c-b106-bfde5128ca5a None admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:25.931002 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c258947b-66d8-463c-b106-bfde5128ca5a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:02:25.931301 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c258947b-66d8-463c-b106-bfde5128ca5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:02:25.932698 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c258947b-66d8-463c-b106-bfde5128ca5a None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:02:25.940319 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 579/1181] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 01:02:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:02:25.946243 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e99474-fe7b-4774-916d-a5f6a292ed76 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:02:25.988653 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:02:26.043821 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e99474-fe7b-4774-916d-a5f6a292ed76 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:02:26.076202 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86390621-a6ae-407f-9fc8-d1f8be0404b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:02:26.076815 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86390621-a6ae-407f-9fc8-d1f8be0404b1 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:26.077093 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86390621-a6ae-407f-9fc8-d1f8be0404b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:26.077367 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86390621-a6ae-407f-9fc8-d1f8be0404b1 None admin] PATH_INFO: `/v3/users/8faad8435adb44aeb440f389ef5a6086` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:26.090538 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86390621-a6ae-407f-9fc8-d1f8be0404b1 None admin] RBAC: Authorizing `identity:get_user(user_id=8faad8435adb44aeb440f389ef5a6086)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:02:26.091966 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86390621-a6ae-407f-9fc8-d1f8be0404b1 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:02:26.093576 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 603/1182] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 8 01:02:25 2022] GET /identity/v3/users/8faad8435adb44aeb440f389ef5a6086 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:02:26.099271 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c258947b-66d8-463c-b106-bfde5128ca5a None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:02:26.140024 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:02:26.197971 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c258947b-66d8-463c-b106-bfde5128ca5a None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:02:26.230542 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a1f59b-a088-41af-8c1c-1dfe0ad077d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:02:26.231129 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1f59b-a088-41af-8c1c-1dfe0ad077d8 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:26.231403 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1f59b-a088-41af-8c1c-1dfe0ad077d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:26.231667 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1f59b-a088-41af-8c1c-1dfe0ad077d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:26.296339 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a1f59b-a088-41af-8c1c-1dfe0ad077d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:02:26.297786 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a1f59b-a088-41af-8c1c-1dfe0ad077d8 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:02:26.336032 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:02:26.393756 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 580/1183] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 01:02:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 01:02:26.399726 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86390621-a6ae-407f-9fc8-d1f8be0404b1 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:02:26.494669 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:02:26.497586 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86390621-a6ae-407f-9fc8-d1f8be0404b1 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:02:26.530822 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de69e334-916e-4f7b-905f-1eb9c01a79c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:02:26.531434 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de69e334-916e-4f7b-905f-1eb9c01a79c8 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:26.531709 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de69e334-916e-4f7b-905f-1eb9c01a79c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:26.531979 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de69e334-916e-4f7b-905f-1eb9c01a79c8 None admin] PATH_INFO: `/v3/projects/55a1dddca12a413294493866d61ae2d6` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:26.533168 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de69e334-916e-4f7b-905f-1eb9c01a79c8 None admin] RBAC: Authorizing `identity:get_project(project_id=55a1dddca12a413294493866d61ae2d6)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:02:26.534581 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de69e334-916e-4f7b-905f-1eb9c01a79c8 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:02:26.536184 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 604/1184] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 01:02:26 2022] GET /identity/v3/projects/55a1dddca12a413294493866d61ae2d6 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:02:26.542469 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a1f59b-a088-41af-8c1c-1dfe0ad077d8 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:02:26.640705 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a1f59b-a088-41af-8c1c-1dfe0ad077d8 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:02:26.674360 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e14b800-92fa-4550-a2dc-f97b36f7f48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:02:26.674948 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14b800-92fa-4550-a2dc-f97b36f7f48d None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:26.675222 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14b800-92fa-4550-a2dc-f97b36f7f48d None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:26.675489 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14b800-92fa-4550-a2dc-f97b36f7f48d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:26.676224 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e14b800-92fa-4550-a2dc-f97b36f7f48d None admin] RBAC: Adding query filter params (role.id=34de369ec7c54715aa83a9052a26c3f2, scope.project.id=55a1dddca12a413294493866d61ae2d6, user.id=8faad8435adb44aeb440f389ef5a6086) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:02:26.676528 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e14b800-92fa-4550-a2dc-f97b36f7f48d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:02:26.677944 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e14b800-92fa-4550-a2dc-f97b36f7f48d None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:02:26.683499 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 581/1185] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 8 01:02:26 2022] GET /identity/v3/role_assignments?user.id=8faad8435adb44aeb440f389ef5a6086&scope.project.id=55a1dddca12a413294493866d61ae2d6&role.id=34de369ec7c54715aa83a9052a26c3f2 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:02:28.795645 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080567ac-cfb9-4d06-9979-f295ad094e24 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:28.796183 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080567ac-cfb9-4d06-9979-f295ad094e24 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:28.796301 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080567ac-cfb9-4d06-9979-f295ad094e24 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:28.797577 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 605/1186] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:02:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:02:28.802672 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f932c82-4b12-4fc8-ad27-8a663e7883ca None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:28.803034 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f932c82-4b12-4fc8-ad27-8a663e7883ca None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:28.803225 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f932c82-4b12-4fc8-ad27-8a663e7883ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:28.811104 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:02:28.822599 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-2f932c82-4b12-4fc8-ad27-8a663e7883ca None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:02:28.823788 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-2f932c82-4b12-4fc8-ad27-8a663e7883ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:02:28.888677 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 582/1187] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:02:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 01:02:29.154361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080567ac-cfb9-4d06-9979-f295ad094e24 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:02:29.252060 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080567ac-cfb9-4d06-9979-f295ad094e24 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:02:29.286976 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef49fa86-a578-4d7f-b39f-0235c401899d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:02:29.287581 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef49fa86-a578-4d7f-b39f-0235c401899d None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:29.287858 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef49fa86-a578-4d7f-b39f-0235c401899d None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:29.288124 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef49fa86-a578-4d7f-b39f-0235c401899d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:29.297828 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef49fa86-a578-4d7f-b39f-0235c401899d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:02:29.302628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-ef49fa86-a578-4d7f-b39f-0235c401899d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 01:02:29.302628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:02:29.302628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:02:29.302628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:02:29.302628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:02:29.302628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:02:29.302628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:02:29.302628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:02:29.302628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:02:29.302628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:02:29.302628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:02:29.302628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:02:29.302628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:02:29.302628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:02:29.302628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:02:29.302628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:02:29.303710 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:02:29.303710 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:02:29.303710 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:02:29.303710 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:02:29.303710 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:02:29.303710 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:02:29.303710 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:02:29.303710 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:02:29.303710 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:02:29.303710 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:02:29.303710 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:02:29.303710 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:02:29.303710 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:02:29.303710 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:02:29.303710 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:02:29.303710 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:02:29.304791 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:02:29.304791 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:02:29.304791 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:02:29.304791 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:02:29.304791 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:02:29.304791 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:02:29.304791 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:02:29.304791 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:02:29.304791 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:02:29.304791 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:02:29.304791 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 01:02:29.304791 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:02:29.304791 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 606/1188] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 8 01:02:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:02:29.309785 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f932c82-4b12-4fc8-ad27-8a663e7883ca None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:02:29.407931 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f932c82-4b12-4fc8-ad27-8a663e7883ca None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:02:29.440694 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef88273-f0d7-456e-8a72-b1fc6e978287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:02:29.441283 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef88273-f0d7-456e-8a72-b1fc6e978287 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:29.441559 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef88273-f0d7-456e-8a72-b1fc6e978287 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:29.441827 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef88273-f0d7-456e-8a72-b1fc6e978287 None admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:29.442476 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef88273-f0d7-456e-8a72-b1fc6e978287 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:02:29.442774 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef88273-f0d7-456e-8a72-b1fc6e978287 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:02:29.444164 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef88273-f0d7-456e-8a72-b1fc6e978287 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:02:29.445449 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:02:29.453393 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 583/1189] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 01:02:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:02:29.459194 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef49fa86-a578-4d7f-b39f-0235c401899d None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:02:29.486781 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:02:29.558021 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef49fa86-a578-4d7f-b39f-0235c401899d None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:02:29.590652 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ed04d4-0c63-4142-a8a6-6bbfd518e002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:02:29.591269 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ed04d4-0c63-4142-a8a6-6bbfd518e002 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:29.591567 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ed04d4-0c63-4142-a8a6-6bbfd518e002 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:29.591821 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ed04d4-0c63-4142-a8a6-6bbfd518e002 None admin] PATH_INFO: `/v3/users/8faad8435adb44aeb440f389ef5a6086` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:29.592959 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ed04d4-0c63-4142-a8a6-6bbfd518e002 None admin] RBAC: Authorizing `identity:get_user(user_id=8faad8435adb44aeb440f389ef5a6086)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:02:29.594389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ed04d4-0c63-4142-a8a6-6bbfd518e002 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:02:29.595944 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 607/1190] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 8 01:02:29 2022] GET /identity/v3/users/8faad8435adb44aeb440f389ef5a6086 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:02:29.601624 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef88273-f0d7-456e-8a72-b1fc6e978287 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:02:29.703026 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef88273-f0d7-456e-8a72-b1fc6e978287 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:02:29.736475 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e431a430-1016-4d58-8fd1-f69e608d93c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:02:29.737099 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431a430-1016-4d58-8fd1-f69e608d93c1 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:29.737375 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431a430-1016-4d58-8fd1-f69e608d93c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:29.737646 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431a430-1016-4d58-8fd1-f69e608d93c1 None admin] PATH_INFO: `/v3/projects/55a1dddca12a413294493866d61ae2d6` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:29.738752 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e431a430-1016-4d58-8fd1-f69e608d93c1 None admin] RBAC: Authorizing `identity:get_project(project_id=55a1dddca12a413294493866d61ae2d6)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:02:29.740235 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e431a430-1016-4d58-8fd1-f69e608d93c1 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:02:29.742035 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 584/1191] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 01:02:29 2022] GET /identity/v3/projects/55a1dddca12a413294493866d61ae2d6 => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:02:29.748547 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ed04d4-0c63-4142-a8a6-6bbfd518e002 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:02:29.842539 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:02:29.849322 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ed04d4-0c63-4142-a8a6-6bbfd518e002 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:02:29.883530 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38feeb28-1aa6-4d58-bc3f-f03370b8ef7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:02:29.884252 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38feeb28-1aa6-4d58-bc3f-f03370b8ef7c None admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:29.884532 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38feeb28-1aa6-4d58-bc3f-f03370b8ef7c None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:29.884806 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38feeb28-1aa6-4d58-bc3f-f03370b8ef7c None admin] PATH_INFO: `/v3/projects/55a1dddca12a413294493866d61ae2d6/users/8faad8435adb44aeb440f389ef5a6086/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:29.886975 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38feeb28-1aa6-4d58-bc3f-f03370b8ef7c None admin] RBAC: Authorizing `identity:create_grant(project_id=55a1dddca12a413294493866d61ae2d6, user_id=8faad8435adb44aeb440f389ef5a6086, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:02:29.888464 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38feeb28-1aa6-4d58-bc3f-f03370b8ef7c None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:02:29.901800 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 608/1192] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 8 01:02:29 2022] PUT /identity/v3/projects/55a1dddca12a413294493866d61ae2d6/users/8faad8435adb44aeb440f389ef5a6086/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:02:32.022522 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de08f7a-7a77-444d-9d5d-65632d8210dd None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:32.023056 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de08f7a-7a77-444d-9d5d-65632d8210dd None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:32.023056 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de08f7a-7a77-444d-9d5d-65632d8210dd None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:32.025193 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 585/1193] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:02:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:02:32.030343 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970ba95-a9e7-4408-945a-f4390c31f7a9 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:32.030782 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970ba95-a9e7-4408-945a-f4390c31f7a9 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:32.030877 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970ba95-a9e7-4408-945a-f4390c31f7a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:32.048771 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-c970ba95-a9e7-4408-945a-f4390c31f7a9 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:02:32.049943 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-c970ba95-a9e7-4408-945a-f4390c31f7a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:02:32.113568 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 609/1194] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:02:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 01:02:32.387044 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de08f7a-7a77-444d-9d5d-65632d8210dd None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:02:32.449952 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:02:32.488608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de08f7a-7a77-444d-9d5d-65632d8210dd None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:02:32.520687 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1f5fd5-3acd-4ae9-82d3-e8c3aa2f19ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:02:32.521300 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f5fd5-3acd-4ae9-82d3-e8c3aa2f19ab None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:32.521578 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f5fd5-3acd-4ae9-82d3-e8c3aa2f19ab None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:32.521851 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f5fd5-3acd-4ae9-82d3-e8c3aa2f19ab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:32.532621 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1f5fd5-3acd-4ae9-82d3-e8c3aa2f19ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:02:32.537365 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-5c1f5fd5-3acd-4ae9-82d3-e8c3aa2f19ab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 01:02:32.537365 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:02:32.537365 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:02:32.537365 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:02:32.537365 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:02:32.537365 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:02:32.537365 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:02:32.537365 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:02:32.537365 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:02:32.537365 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:02:32.537365 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:02:32.537365 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:02:32.537365 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:02:32.537365 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:02:32.537365 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:02:32.537365 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:02:32.538474 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:02:32.538474 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:02:32.538474 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:02:32.538474 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:02:32.538474 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:02:32.538474 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:02:32.538474 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:02:32.538474 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:02:32.538474 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:02:32.538474 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:02:32.538474 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:02:32.538474 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:02:32.538474 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:02:32.538474 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:02:32.538474 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:02:32.538474 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:02:32.539385 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:02:32.539385 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:02:32.539385 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:02:32.539385 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:02:32.539385 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:02:32.539385 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:02:32.539385 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:02:32.539385 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:02:32.539385 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:02:32.539385 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:02:32.539385 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 01:02:32.539385 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:02:32.539385 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 586/1195] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 8 01:02:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:02:32.544476 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c970ba95-a9e7-4408-945a-f4390c31f7a9 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:02:32.624579 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:02:32.641458 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c970ba95-a9e7-4408-945a-f4390c31f7a9 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:02:32.675253 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e83a577-c2c4-4985-9811-16f60826620b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:02:32.675851 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e83a577-c2c4-4985-9811-16f60826620b None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:32.676129 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e83a577-c2c4-4985-9811-16f60826620b None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:32.676434 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e83a577-c2c4-4985-9811-16f60826620b None admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:32.677089 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e83a577-c2c4-4985-9811-16f60826620b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:02:32.677388 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e83a577-c2c4-4985-9811-16f60826620b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:02:32.678742 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e83a577-c2c4-4985-9811-16f60826620b None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:02:32.687048 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 610/1196] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 01:02:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:02:32.692900 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1f5fd5-3acd-4ae9-82d3-e8c3aa2f19ab None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:02:32.789942 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1f5fd5-3acd-4ae9-82d3-e8c3aa2f19ab None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:02:32.812071 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:02:32.823313 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d2bf9e-b54c-48c8-9bfa-e7f60edd2d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:02:32.823929 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d2bf9e-b54c-48c8-9bfa-e7f60edd2d6e None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:32.824239 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d2bf9e-b54c-48c8-9bfa-e7f60edd2d6e None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:32.824513 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d2bf9e-b54c-48c8-9bfa-e7f60edd2d6e None admin] PATH_INFO: `/v3/users/8faad8435adb44aeb440f389ef5a6086` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:32.825618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d2bf9e-b54c-48c8-9bfa-e7f60edd2d6e None admin] RBAC: Authorizing `identity:get_user(user_id=8faad8435adb44aeb440f389ef5a6086)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:02:32.827056 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d2bf9e-b54c-48c8-9bfa-e7f60edd2d6e None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:02:32.828644 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 587/1197] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Sep 8 01:02:32 2022] GET /identity/v3/users/8faad8435adb44aeb440f389ef5a6086 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:02:32.834327 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e83a577-c2c4-4985-9811-16f60826620b None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:02:32.932392 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e83a577-c2c4-4985-9811-16f60826620b None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:02:32.965008 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17b033c-0c91-4194-b9f0-0befe40b13d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:02:32.965603 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b033c-0c91-4194-b9f0-0befe40b13d7 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:32.965880 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b033c-0c91-4194-b9f0-0befe40b13d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:32.966147 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b033c-0c91-4194-b9f0-0befe40b13d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:32.983533 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:02:33.032374 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17b033c-0c91-4194-b9f0-0befe40b13d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:02:33.033821 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17b033c-0c91-4194-b9f0-0befe40b13d7 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:02:33.129345 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 611/1198] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 01:02:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 01:02:33.135929 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d2bf9e-b54c-48c8-9bfa-e7f60edd2d6e None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:02:33.237040 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d2bf9e-b54c-48c8-9bfa-e7f60edd2d6e None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:02:33.269471 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2189a2a-5402-4ad9-acbc-2a7c154061b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:02:33.270085 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2189a2a-5402-4ad9-acbc-2a7c154061b7 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:33.270366 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2189a2a-5402-4ad9-acbc-2a7c154061b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:33.270642 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2189a2a-5402-4ad9-acbc-2a7c154061b7 None admin] PATH_INFO: `/v3/projects/55a1dddca12a413294493866d61ae2d6` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:33.271747 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2189a2a-5402-4ad9-acbc-2a7c154061b7 None admin] RBAC: Authorizing `identity:get_project(project_id=55a1dddca12a413294493866d61ae2d6)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:02:33.273226 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2189a2a-5402-4ad9-acbc-2a7c154061b7 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:02:33.275359 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 588/1199] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Sep 8 01:02:33 2022] GET /identity/v3/projects/55a1dddca12a413294493866d61ae2d6 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:02:33.281438 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17b033c-0c91-4194-b9f0-0befe40b13d7 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:02:33.329253 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:02:33.379199 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17b033c-0c91-4194-b9f0-0befe40b13d7 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:02:33.413320 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d44000-6782-4861-ab1c-f25f5c855ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:02:33.413918 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d44000-6782-4861-ab1c-f25f5c855ef5 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:02:33.414251 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d44000-6782-4861-ab1c-f25f5c855ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:02:33.414703 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d44000-6782-4861-ab1c-f25f5c855ef5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:02:33.415428 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d44000-6782-4861-ab1c-f25f5c855ef5 None admin] RBAC: Adding query filter params (role.id=34de369ec7c54715aa83a9052a26c3f2, scope.project.id=55a1dddca12a413294493866d61ae2d6, user.id=8faad8435adb44aeb440f389ef5a6086) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:02:33.415740 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d44000-6782-4861-ab1c-f25f5c855ef5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:02:33.417199 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d44000-6782-4861-ab1c-f25f5c855ef5 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:02:33.422839 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 612/1200] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 8 01:02:33 2022] GET /identity/v3/role_assignments?user.id=8faad8435adb44aeb440f389ef5a6086&scope.project.id=55a1dddca12a413294493866d61ae2d6&role.id=34de369ec7c54715aa83a9052a26c3f2 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:02:43.265428 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=70 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:02:43.420272 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=44 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:29.130702 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442035f0-7643-4845-8ff4-dcbbdd679c3f None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:29.131857 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442035f0-7643-4845-8ff4-dcbbdd679c3f None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:29.131857 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442035f0-7643-4845-8ff4-dcbbdd679c3f None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:29.133524 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 589/1201] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:03:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:03:29.139321 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e55886-4ac6-42a9-b017-d7e9f74fed9c None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:29.139781 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e55886-4ac6-42a9-b017-d7e9f74fed9c None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:29.139866 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e55886-4ac6-42a9-b017-d7e9f74fed9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:29.162764 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-83e55886-4ac6-42a9-b017-d7e9f74fed9c None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:03:29.164277 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-83e55886-4ac6-42a9-b017-d7e9f74fed9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:03:29.271323 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 613/1202] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:03:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 01:03:29.560450 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442035f0-7643-4845-8ff4-dcbbdd679c3f None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:29.697789 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442035f0-7643-4845-8ff4-dcbbdd679c3f None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:29.741978 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ca298c-9437-44f2-a8d4-91f50d9a2a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:29.742611 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ca298c-9437-44f2-a8d4-91f50d9a2a15 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:29.742890 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ca298c-9437-44f2-a8d4-91f50d9a2a15 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:29.743161 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ca298c-9437-44f2-a8d4-91f50d9a2a15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:29.762250 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ca298c-9437-44f2-a8d4-91f50d9a2a15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:29.764546 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ca298c-9437-44f2-a8d4-91f50d9a2a15 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:29.786983 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-02ca298c-9437-44f2-a8d4-91f50d9a2a15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:03:29.786983 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:03:29.786983 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:03:29.786983 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:03:29.786983 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:03:29.786983 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:03:29.786983 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:03:29.786983 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:03:29.786983 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:03:29.786983 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:03:29.786983 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:03:29.786983 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:03:29.786983 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:03:29.786983 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:03:29.786983 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:03:29.786983 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:03:29.789820 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:03:29.789820 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:03:29.789820 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:03:29.789820 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:03:29.789820 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:03:29.789820 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:03:29.789820 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:03:29.789820 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:03:29.789820 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:03:29.789820 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:03:29.789820 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:03:29.789820 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:03:29.789820 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:03:29.789820 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:03:29.789820 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:03:29.789820 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:03:29.789820 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:03:29.791668 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:03:29.791668 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:03:29.791668 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:03:29.794188 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 590/1203] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:03:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:03:29.800857 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e55886-4ac6-42a9-b017-d7e9f74fed9c None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:29.953246 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e55886-4ac6-42a9-b017-d7e9f74fed9c None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:29.995030 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:30.011520 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef10a8-99ff-4ffe-9f39-d63cb48142fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:30.016447 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef10a8-99ff-4ffe-9f39-d63cb48142fb None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:30.016905 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef10a8-99ff-4ffe-9f39-d63cb48142fb None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:30.017385 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef10a8-99ff-4ffe-9f39-d63cb48142fb None admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:30.018206 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeef10a8-99ff-4ffe-9f39-d63cb48142fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:03:30.018921 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeef10a8-99ff-4ffe-9f39-d63cb48142fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:30.020606 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeef10a8-99ff-4ffe-9f39-d63cb48142fb None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:30.040735 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 614/1204] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:03:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:03:30.049731 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ca298c-9437-44f2-a8d4-91f50d9a2a15 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:30.132099 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:30.193507 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ca298c-9437-44f2-a8d4-91f50d9a2a15 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:30.247708 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b760721e-73a2-4b02-ac35-4821d135a1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:30.248435 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b760721e-73a2-4b02-ac35-4821d135a1a8 None admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:30.248721 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b760721e-73a2-4b02-ac35-4821d135a1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:30.249026 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b760721e-73a2-4b02-ac35-4821d135a1a8 None admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:30.249856 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b760721e-73a2-4b02-ac35-4821d135a1a8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:30.251417 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b760721e-73a2-4b02-ac35-4821d135a1a8 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:30.288344 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 591/1205] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Sep 8 01:03:30 2022] POST /identity/v3/users => generated 251 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:03:32.748105 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758fa30-4e17-4e34-810f-a91f72dde2c7 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:32.748730 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758fa30-4e17-4e34-810f-a91f72dde2c7 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:32.748730 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758fa30-4e17-4e34-810f-a91f72dde2c7 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:32.750019 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 615/1206] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:03:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:03:32.755285 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8d6f2a-0992-4c2c-ba2c-2416790d9ef8 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:32.755701 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8d6f2a-0992-4c2c-ba2c-2416790d9ef8 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:32.756411 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8d6f2a-0992-4c2c-ba2c-2416790d9ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:32.779651 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-9a8d6f2a-0992-4c2c-ba2c-2416790d9ef8 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:03:32.780894 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-9a8d6f2a-0992-4c2c-ba2c-2416790d9ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:03:32.854173 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 592/1207] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:03:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 01:03:33.219969 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0758fa30-4e17-4e34-810f-a91f72dde2c7 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:33.352084 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0758fa30-4e17-4e34-810f-a91f72dde2c7 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:33.408764 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed93c35-ea32-493c-8fde-dd10a311fa9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:33.409613 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed93c35-ea32-493c-8fde-dd10a311fa9d None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:33.409933 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed93c35-ea32-493c-8fde-dd10a311fa9d None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:33.410344 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed93c35-ea32-493c-8fde-dd10a311fa9d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:33.425935 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed93c35-ea32-493c-8fde-dd10a311fa9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:03:33.433403 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-2ed93c35-ea32-493c-8fde-dd10a311fa9d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 01:03:33.433403 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:03:33.433403 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:03:33.433403 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:03:33.433403 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:03:33.433403 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:03:33.433403 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:03:33.433403 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:03:33.433403 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:03:33.433403 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:03:33.433403 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:03:33.433403 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:03:33.433403 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:03:33.433403 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:03:33.433403 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:03:33.433403 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:03:33.434420 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:03:33.434420 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:03:33.434420 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:03:33.434420 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:03:33.434420 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:03:33.434420 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:03:33.434420 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:03:33.434420 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:03:33.434420 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:03:33.434420 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:03:33.434420 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:03:33.434420 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:03:33.434420 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:03:33.434420 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:03:33.434420 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:03:33.434420 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:03:33.435276 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:03:33.435276 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:03:33.435276 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:03:33.435276 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:03:33.435276 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:03:33.435276 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:03:33.435276 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:03:33.435276 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:03:33.435276 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:03:33.435276 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:03:33.435276 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 01:03:33.435276 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:03:33.435276 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 616/1208] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 8 01:03:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:03:33.442494 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8d6f2a-0992-4c2c-ba2c-2416790d9ef8 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:33.488189 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:33.567357 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8d6f2a-0992-4c2c-ba2c-2416790d9ef8 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:33.607677 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcabca1-5d37-42f7-a1e7-ae0aeeafeab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:33.608423 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcabca1-5d37-42f7-a1e7-ae0aeeafeab6 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:33.608779 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcabca1-5d37-42f7-a1e7-ae0aeeafeab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:33.609075 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcabca1-5d37-42f7-a1e7-ae0aeeafeab6 None admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:33.609939 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcabca1-5d37-42f7-a1e7-ae0aeeafeab6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:03:33.610198 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcabca1-5d37-42f7-a1e7-ae0aeeafeab6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:33.611610 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcabca1-5d37-42f7-a1e7-ae0aeeafeab6 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:33.622974 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 593/1209] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 8 01:03:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:03:33.629163 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed93c35-ea32-493c-8fde-dd10a311fa9d None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:33.700331 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:33.769775 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed93c35-ea32-493c-8fde-dd10a311fa9d None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:33.809649 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c158c50b-4d85-4572-9a5f-77f5a497332c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:33.810296 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158c50b-4d85-4572-9a5f-77f5a497332c None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:33.810579 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158c50b-4d85-4572-9a5f-77f5a497332c None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:33.810849 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158c50b-4d85-4572-9a5f-77f5a497332c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:33.822466 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c158c50b-4d85-4572-9a5f-77f5a497332c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:33.823985 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c158c50b-4d85-4572-9a5f-77f5a497332c None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:33.841240 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-c158c50b-4d85-4572-9a5f-77f5a497332c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:03:33.841240 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:03:33.841240 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:03:33.841240 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:03:33.841240 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:03:33.841240 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:03:33.841240 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:03:33.841240 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:03:33.841240 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:03:33.841240 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:03:33.841240 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:03:33.841240 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:03:33.841240 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:03:33.841240 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:03:33.841240 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:03:33.841240 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:03:33.842366 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:03:33.842366 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:03:33.842366 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:03:33.842366 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:03:33.842366 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:03:33.842366 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:03:33.842366 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:03:33.842366 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:03:33.842366 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:03:33.842366 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:03:33.842366 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:03:33.842366 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:03:33.842366 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:03:33.842366 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:03:33.842366 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:03:33.842366 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:03:33.842366 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:03:33.843278 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:03:33.843278 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:03:33.843278 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:03:33.847519 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 617/1210] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:03:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:03:33.848371 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcabca1-5d37-42f7-a1e7-ae0aeeafeab6 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:33.960928 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:33.965094 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcabca1-5d37-42f7-a1e7-ae0aeeafeab6 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:34.012926 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f7a852-90e0-4d0e-8b7f-a18d823a550c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:34.013564 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f7a852-90e0-4d0e-8b7f-a18d823a550c None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:34.013848 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f7a852-90e0-4d0e-8b7f-a18d823a550c None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:34.014124 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f7a852-90e0-4d0e-8b7f-a18d823a550c None admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:34.014787 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f7a852-90e0-4d0e-8b7f-a18d823a550c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:03:34.015209 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f7a852-90e0-4d0e-8b7f-a18d823a550c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:34.017608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f7a852-90e0-4d0e-8b7f-a18d823a550c None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:34.036463 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 594/1211] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:03:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:03:34.043188 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c158c50b-4d85-4572-9a5f-77f5a497332c None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:34.155826 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c158c50b-4d85-4572-9a5f-77f5a497332c None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:34.157516 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:34.192873 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e108f3be-342b-49ec-bd69-359807a0266f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:34.194099 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108f3be-342b-49ec-bd69-359807a0266f None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:34.194382 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108f3be-342b-49ec-bd69-359807a0266f None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:34.194654 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108f3be-342b-49ec-bd69-359807a0266f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:34.202899 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e108f3be-342b-49ec-bd69-359807a0266f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:34.204389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e108f3be-342b-49ec-bd69-359807a0266f None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:34.216029 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-e108f3be-342b-49ec-bd69-359807a0266f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 08 01:03:34.216029 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:03:34.216029 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:03:34.216029 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:03:34.216029 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:03:34.216029 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:03:34.216029 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:03:34.216029 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:03:34.216029 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:03:34.216029 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:03:34.216029 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:03:34.216029 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:03:34.216029 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 01:03:34.216029 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 01:03:34.216029 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 01:03:34.216029 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 01:03:34.217111 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:03:34.217111 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:03:34.217111 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 01:03:34.217111 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:03:34.217111 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 01:03:34.217111 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:03:34.217111 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:03:34.217111 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:03:34.217111 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:03:34.217111 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:03:34.217111 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:03:34.217111 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:03:34.217111 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:03:34.217111 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:03:34.217111 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:03:34.217111 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:03:34.217111 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:03:34.217111 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:03:34.223308 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:03:34.223308 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:03:34.223308 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 01:03:34.223308 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 01:03:34.223308 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 01:03:34.223308 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 01:03:34.223308 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 01:03:34.223308 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 01:03:34.223308 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 08 01:03:34.223308 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:03:34.223308 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 618/1212] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:03:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:03:34.223923 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f7a852-90e0-4d0e-8b7f-a18d823a550c None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:34.333364 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f7a852-90e0-4d0e-8b7f-a18d823a550c None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:34.370798 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecda273-dfc3-4683-a9e6-b1cf41886d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:34.371416 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecda273-dfc3-4683-a9e6-b1cf41886d52 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:34.371695 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecda273-dfc3-4683-a9e6-b1cf41886d52 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:34.371964 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecda273-dfc3-4683-a9e6-b1cf41886d52 None admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:34.372727 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecda273-dfc3-4683-a9e6-b1cf41886d52 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:03:34.373031 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecda273-dfc3-4683-a9e6-b1cf41886d52 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:34.374478 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecda273-dfc3-4683-a9e6-b1cf41886d52 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:34.393550 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 595/1213] 10.222.0.42 () {56 vars in 1143 bytes} [Thu Sep 8 01:03:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:03:34.401462 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e108f3be-342b-49ec-bd69-359807a0266f None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:34.509754 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e108f3be-342b-49ec-bd69-359807a0266f None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:34.545745 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8940cf-b29e-4a86-88ea-b808fef3308e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:34.546348 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8940cf-b29e-4a86-88ea-b808fef3308e None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:34.546626 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8940cf-b29e-4a86-88ea-b808fef3308e None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:34.546900 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8940cf-b29e-4a86-88ea-b808fef3308e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:34.581337 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:34.629281 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8940cf-b29e-4a86-88ea-b808fef3308e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:34.631679 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8940cf-b29e-4a86-88ea-b808fef3308e None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:34.746415 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 619/1214] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 01:03:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 01:03:34.747921 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecda273-dfc3-4683-a9e6-b1cf41886d52 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:34.769079 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:34.870090 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecda273-dfc3-4683-a9e6-b1cf41886d52 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:34.910038 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff90366-a056-4a08-9906-3ccdfb8b8fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:34.910602 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff90366-a056-4a08-9906-3ccdfb8b8fd1 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:34.910881 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff90366-a056-4a08-9906-3ccdfb8b8fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:34.911156 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff90366-a056-4a08-9906-3ccdfb8b8fd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:34.924077 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff90366-a056-4a08-9906-3ccdfb8b8fd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:34.925676 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff90366-a056-4a08-9906-3ccdfb8b8fd1 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:34.943818 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-1ff90366-a056-4a08-9906-3ccdfb8b8fd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:03:34.943818 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:03:34.943818 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:03:34.943818 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:03:34.943818 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:03:34.943818 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:03:34.943818 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:03:34.943818 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:03:34.943818 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:03:34.943818 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:03:34.943818 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:03:34.943818 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:03:34.943818 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:03:34.943818 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:03:34.943818 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:03:34.943818 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:03:34.945166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:03:34.945166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:03:34.945166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:03:34.945166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:03:34.945166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:03:34.945166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:03:34.945166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:03:34.945166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:03:34.945166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:03:34.945166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:03:34.945166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:03:34.945166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:03:34.945166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:03:34.945166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:03:34.945166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:03:34.945166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:03:34.945166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:03:34.946061 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:03:34.946061 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:03:34.946061 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:03:34.946061 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 596/1215] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:03:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:03:34.951059 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8940cf-b29e-4a86-88ea-b808fef3308e None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:35.036289 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:35.079180 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8940cf-b29e-4a86-88ea-b808fef3308e None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:35.123701 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef19718-6ab1-458b-82cf-fa3bddd71a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:35.124339 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef19718-6ab1-458b-82cf-fa3bddd71a6e None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:35.124661 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef19718-6ab1-458b-82cf-fa3bddd71a6e None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:35.124891 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef19718-6ab1-458b-82cf-fa3bddd71a6e None admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:35.125520 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef19718-6ab1-458b-82cf-fa3bddd71a6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:03:35.125894 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef19718-6ab1-458b-82cf-fa3bddd71a6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:35.127379 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef19718-6ab1-458b-82cf-fa3bddd71a6e None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:35.142233 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 620/1216] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:03:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:03:35.146407 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff90366-a056-4a08-9906-3ccdfb8b8fd1 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:35.228161 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:35.268964 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff90366-a056-4a08-9906-3ccdfb8b8fd1 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:35.321833 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b235fc-407a-4bde-835d-ccc2ea7a04c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:35.322533 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b235fc-407a-4bde-835d-ccc2ea7a04c5 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:35.322811 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b235fc-407a-4bde-835d-ccc2ea7a04c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:35.323049 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b235fc-407a-4bde-835d-ccc2ea7a04c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:35.331851 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b235fc-407a-4bde-835d-ccc2ea7a04c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:35.333624 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b235fc-407a-4bde-835d-ccc2ea7a04c5 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:35.347639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-86b235fc-407a-4bde-835d-ccc2ea7a04c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:03:35.347639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:03:35.347639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:03:35.347639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:03:35.347639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:03:35.347639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:03:35.347639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:03:35.347639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:03:35.347639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:03:35.347639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:03:35.347639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:03:35.347639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:03:35.347639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 01:03:35.347639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 01:03:35.347639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 01:03:35.348736 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 01:03:35.348736 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:03:35.348736 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:03:35.348736 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:03:35.348736 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:03:35.348736 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:03:35.348736 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:03:35.348736 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:03:35.348736 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:03:35.348736 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:03:35.348736 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:03:35.348736 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:03:35.348736 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:03:35.348736 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:03:35.348736 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:03:35.348736 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:03:35.348736 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:03:35.349785 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 01:03:35.349785 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 01:03:35.349785 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 01:03:35.349785 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 01:03:35.349785 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 01:03:35.349785 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 01:03:35.349785 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:03:35.349785 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:03:35.349785 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 597/1217] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 01:03:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:03:35.355267 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef19718-6ab1-458b-82cf-fa3bddd71a6e None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:35.476553 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef19718-6ab1-458b-82cf-fa3bddd71a6e None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:35.500571 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:35.515088 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfef6fd-c4aa-4999-afce-3646598bf894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:35.516050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfef6fd-c4aa-4999-afce-3646598bf894 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:35.516437 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfef6fd-c4aa-4999-afce-3646598bf894 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:35.516915 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfef6fd-c4aa-4999-afce-3646598bf894 None admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:35.517982 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfef6fd-c4aa-4999-afce-3646598bf894 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:03:35.518327 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfef6fd-c4aa-4999-afce-3646598bf894 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:35.520057 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfef6fd-c4aa-4999-afce-3646598bf894 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:35.535644 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 621/1218] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 8 01:03:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:03:35.542569 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b235fc-407a-4bde-835d-ccc2ea7a04c5 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:35.661570 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b235fc-407a-4bde-835d-ccc2ea7a04c5 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:35.696070 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:35.702948 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc274ca2-43f6-4e70-9d1e-c7bb8cf0a86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:35.703552 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc274ca2-43f6-4e70-9d1e-c7bb8cf0a86c None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:35.703827 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc274ca2-43f6-4e70-9d1e-c7bb8cf0a86c None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:35.704098 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc274ca2-43f6-4e70-9d1e-c7bb8cf0a86c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:35.704847 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc274ca2-43f6-4e70-9d1e-c7bb8cf0a86c None admin] RBAC: Adding query filter params (role.id=609d73f56e6a48698397131168abe6df, scope.project.id=78ad2a8fb5264fb4af935e039d28c8de, user.id=301d30bbd3de4cc69e96bb12729442c6) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:03:35.705154 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc274ca2-43f6-4e70-9d1e-c7bb8cf0a86c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:35.706620 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc274ca2-43f6-4e70-9d1e-c7bb8cf0a86c None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:35.712929 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 598/1219] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 8 01:03:35 2022] GET /identity/v3/role_assignments?user.id=301d30bbd3de4cc69e96bb12729442c6&scope.project.id=78ad2a8fb5264fb4af935e039d28c8de&role.id=609d73f56e6a48698397131168abe6df => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:03:38.015669 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae229a0-9a27-4d36-804f-e8c01f62a21d None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:38.016387 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae229a0-9a27-4d36-804f-e8c01f62a21d None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:38.016387 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae229a0-9a27-4d36-804f-e8c01f62a21d None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:38.017884 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 622/1220] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:03:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:03:38.024078 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f411fb0e-2f21-447a-84dd-e15fff622819 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:38.024566 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f411fb0e-2f21-447a-84dd-e15fff622819 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:38.024674 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f411fb0e-2f21-447a-84dd-e15fff622819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:38.045507 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-f411fb0e-2f21-447a-84dd-e15fff622819 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:03:38.046953 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-f411fb0e-2f21-447a-84dd-e15fff622819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:03:38.124453 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 599/1221] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:03:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 01:03:38.446044 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae229a0-9a27-4d36-804f-e8c01f62a21d None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:38.564443 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae229a0-9a27-4d36-804f-e8c01f62a21d None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:38.606415 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c94d06-31ca-4208-91b8-c02a3226261d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:38.607044 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c94d06-31ca-4208-91b8-c02a3226261d None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:38.607323 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c94d06-31ca-4208-91b8-c02a3226261d None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:38.607595 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c94d06-31ca-4208-91b8-c02a3226261d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:38.618837 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c94d06-31ca-4208-91b8-c02a3226261d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:03:38.623635 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-70c94d06-31ca-4208-91b8-c02a3226261d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 01:03:38.623635 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:03:38.623635 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:03:38.623635 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:03:38.623635 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:03:38.623635 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:03:38.623635 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:03:38.623635 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:03:38.623635 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:03:38.623635 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:03:38.623635 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:03:38.623635 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:03:38.623635 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:03:38.623635 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:03:38.623635 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:03:38.623635 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:03:38.624781 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:03:38.624781 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:03:38.624781 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:03:38.624781 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:03:38.624781 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:03:38.624781 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:03:38.624781 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:03:38.624781 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:03:38.624781 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:03:38.624781 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:03:38.624781 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:03:38.624781 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:03:38.624781 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:03:38.624781 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:03:38.624781 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:03:38.624781 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:03:38.625628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:03:38.625628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:03:38.625628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:03:38.625628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:03:38.625628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:03:38.625628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:03:38.625628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:03:38.625628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:03:38.625628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:03:38.625628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:03:38.625628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 01:03:38.625628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:03:38.625628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 623/1222] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 8 01:03:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:03:38.630873 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f411fb0e-2f21-447a-84dd-e15fff622819 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:38.738837 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f411fb0e-2f21-447a-84dd-e15fff622819 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:38.773181 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:38.789118 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ec6ce-cf09-4dd4-aac5-c6cf4f0768d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:38.789875 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ec6ce-cf09-4dd4-aac5-c6cf4f0768d7 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:38.790279 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ec6ce-cf09-4dd4-aac5-c6cf4f0768d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:38.790742 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ec6ce-cf09-4dd4-aac5-c6cf4f0768d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:38.791578 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82ec6ce-cf09-4dd4-aac5-c6cf4f0768d7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:03:38.792010 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82ec6ce-cf09-4dd4-aac5-c6cf4f0768d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:38.793851 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82ec6ce-cf09-4dd4-aac5-c6cf4f0768d7 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:38.814475 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 600/1223] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 8 01:03:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:03:38.816004 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c94d06-31ca-4208-91b8-c02a3226261d None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:38.867635 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:38.941271 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c94d06-31ca-4208-91b8-c02a3226261d None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:38.986592 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4000e359-d771-47dc-b392-1d50965357d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:38.987348 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000e359-d771-47dc-b392-1d50965357d1 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:38.987729 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000e359-d771-47dc-b392-1d50965357d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:38.988099 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000e359-d771-47dc-b392-1d50965357d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:39.002310 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4000e359-d771-47dc-b392-1d50965357d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:39.003788 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4000e359-d771-47dc-b392-1d50965357d1 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:39.022580 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-4000e359-d771-47dc-b392-1d50965357d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:03:39.022580 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:03:39.022580 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:03:39.022580 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:03:39.022580 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:03:39.022580 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:03:39.022580 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:03:39.022580 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:03:39.022580 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:03:39.022580 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:03:39.022580 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:03:39.022580 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:03:39.022580 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:03:39.022580 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:03:39.022580 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:03:39.022580 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:03:39.024255 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:03:39.024255 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:03:39.024255 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:03:39.024255 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:03:39.024255 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:03:39.024255 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:03:39.024255 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:03:39.024255 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:03:39.024255 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:03:39.024255 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:03:39.024255 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:03:39.024255 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:03:39.024255 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:03:39.024255 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:03:39.024255 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:03:39.024255 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:03:39.024255 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:03:39.025159 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:03:39.025159 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:03:39.025159 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:03:39.025159 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 624/1224] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:03:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 01:03:39.029972 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ec6ce-cf09-4dd4-aac5-c6cf4f0768d7 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:39.140811 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ec6ce-cf09-4dd4-aac5-c6cf4f0768d7 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:39.181238 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c35a4a9-090f-4146-a9d5-62910daffc69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:39.181916 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c35a4a9-090f-4146-a9d5-62910daffc69 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:39.182236 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c35a4a9-090f-4146-a9d5-62910daffc69 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:39.182503 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c35a4a9-090f-4146-a9d5-62910daffc69 None admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:39.183236 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c35a4a9-090f-4146-a9d5-62910daffc69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:03:39.183543 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c35a4a9-090f-4146-a9d5-62910daffc69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:39.185092 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c35a4a9-090f-4146-a9d5-62910daffc69 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:39.206364 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 601/1225] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:03:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:03:39.207623 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4000e359-d771-47dc-b392-1d50965357d1 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:39.310944 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:39.318235 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4000e359-d771-47dc-b392-1d50965357d1 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:39.354524 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3eba94-d778-4a28-9777-e1182cc365ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:39.355147 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3eba94-d778-4a28-9777-e1182cc365ce None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:39.355485 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3eba94-d778-4a28-9777-e1182cc365ce None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:39.355758 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3eba94-d778-4a28-9777-e1182cc365ce None admin] PATH_INFO: `/v3/users/placement` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:39.363651 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3eba94-d778-4a28-9777-e1182cc365ce None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:39.365377 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3eba94-d778-4a28-9777-e1182cc365ce None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:39.377810 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-6d3eba94-d778-4a28-9777-e1182cc365ce None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 08 01:03:39.377810 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:03:39.377810 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:03:39.377810 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:03:39.377810 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:03:39.377810 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:03:39.377810 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:03:39.377810 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:03:39.377810 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:03:39.377810 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:03:39.377810 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:03:39.377810 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:03:39.377810 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 01:03:39.377810 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 01:03:39.377810 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 01:03:39.377810 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 01:03:39.378994 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:03:39.378994 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:03:39.378994 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 01:03:39.378994 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:03:39.378994 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 01:03:39.378994 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:03:39.378994 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:03:39.378994 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:03:39.378994 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:03:39.378994 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:03:39.378994 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:03:39.378994 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:03:39.378994 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:03:39.378994 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:03:39.378994 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:03:39.378994 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:03:39.378994 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:03:39.378994 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:03:39.380354 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:03:39.380354 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:03:39.380354 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 01:03:39.380354 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 01:03:39.380354 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 01:03:39.380354 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 01:03:39.380354 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 01:03:39.380354 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 01:03:39.380354 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 08 01:03:39.380354 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:03:39.380354 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 625/1226] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:03:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:03:39.385402 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c35a4a9-090f-4146-a9d5-62910daffc69 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:39.435776 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:39.493611 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c35a4a9-090f-4146-a9d5-62910daffc69 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:39.531339 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2023e118-698b-4f54-8de3-ebf056846927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:39.532076 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2023e118-698b-4f54-8de3-ebf056846927 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:39.532410 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2023e118-698b-4f54-8de3-ebf056846927 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:39.532800 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2023e118-698b-4f54-8de3-ebf056846927 None admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:39.533623 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2023e118-698b-4f54-8de3-ebf056846927 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:03:39.533955 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2023e118-698b-4f54-8de3-ebf056846927 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:39.535554 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2023e118-698b-4f54-8de3-ebf056846927 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:39.561145 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 602/1227] 10.222.0.42 () {56 vars in 1143 bytes} [Thu Sep 8 01:03:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:03:39.562194 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3eba94-d778-4a28-9777-e1182cc365ce None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:39.684985 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3eba94-d778-4a28-9777-e1182cc365ce None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:39.720009 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee4068-b94e-4132-bf92-d3d96a2af8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:39.720743 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee4068-b94e-4132-bf92-d3d96a2af8a3 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:39.721015 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee4068-b94e-4132-bf92-d3d96a2af8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:39.721290 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee4068-b94e-4132-bf92-d3d96a2af8a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:39.732604 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eee4068-b94e-4132-bf92-d3d96a2af8a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:39.734042 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eee4068-b94e-4132-bf92-d3d96a2af8a3 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:39.748794 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-4eee4068-b94e-4132-bf92-d3d96a2af8a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:03:39.748794 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:03:39.748794 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:03:39.748794 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:03:39.748794 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:03:39.748794 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:03:39.748794 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:03:39.748794 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:03:39.748794 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:03:39.748794 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:03:39.748794 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:03:39.748794 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:03:39.748794 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:03:39.748794 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:03:39.748794 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:03:39.748794 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:03:39.749947 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:03:39.749947 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:03:39.749947 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:03:39.749947 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:03:39.749947 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:03:39.749947 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:03:39.749947 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:03:39.749947 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:03:39.749947 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:03:39.749947 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:03:39.749947 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:03:39.749947 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:03:39.749947 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:03:39.749947 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:03:39.749947 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:03:39.749947 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:03:39.749947 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:03:39.751076 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:03:39.751076 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:03:39.751076 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:03:39.751076 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 626/1228] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:03:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 01:03:39.755726 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2023e118-698b-4f54-8de3-ebf056846927 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:39.871137 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2023e118-698b-4f54-8de3-ebf056846927 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:39.878929 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:39.910561 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48855df0-5b54-4f5e-8f30-2542b01060bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:39.911169 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48855df0-5b54-4f5e-8f30-2542b01060bd None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:39.911449 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48855df0-5b54-4f5e-8f30-2542b01060bd None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:39.911717 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48855df0-5b54-4f5e-8f30-2542b01060bd None admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:39.913096 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48855df0-5b54-4f5e-8f30-2542b01060bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:03:39.913387 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48855df0-5b54-4f5e-8f30-2542b01060bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:39.914766 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48855df0-5b54-4f5e-8f30-2542b01060bd None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:39.928769 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 603/1229] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:03:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:03:39.934752 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee4068-b94e-4132-bf92-d3d96a2af8a3 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:39.946598 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:40.046554 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee4068-b94e-4132-bf92-d3d96a2af8a3 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:40.081065 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b3b1a0-e34a-41f5-a7e2-c7aaa00b045b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:40.081697 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3b1a0-e34a-41f5-a7e2-c7aaa00b045b None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:40.082001 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3b1a0-e34a-41f5-a7e2-c7aaa00b045b None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:40.082245 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3b1a0-e34a-41f5-a7e2-c7aaa00b045b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:40.090457 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b3b1a0-e34a-41f5-a7e2-c7aaa00b045b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:40.092005 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b3b1a0-e34a-41f5-a7e2-c7aaa00b045b None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:40.104332 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-09b3b1a0-e34a-41f5-a7e2-c7aaa00b045b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:03:40.104332 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:03:40.104332 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:03:40.104332 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:03:40.104332 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:03:40.104332 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:03:40.104332 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:03:40.104332 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:03:40.104332 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:03:40.104332 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:03:40.104332 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:03:40.104332 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:03:40.104332 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 01:03:40.104332 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 01:03:40.104332 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 01:03:40.105425 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 01:03:40.105425 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:03:40.105425 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:03:40.105425 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:03:40.105425 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:03:40.105425 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:03:40.105425 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:03:40.105425 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:03:40.105425 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:03:40.105425 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:03:40.105425 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:03:40.105425 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:03:40.105425 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:03:40.105425 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:03:40.105425 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:03:40.105425 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:03:40.105425 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:03:40.106359 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 01:03:40.106359 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 01:03:40.106359 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 01:03:40.106359 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 01:03:40.106359 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 01:03:40.106359 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 01:03:40.106359 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:03:40.106359 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:03:40.106359 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 627/1230] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 01:03:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:03:40.111409 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48855df0-5b54-4f5e-8f30-2542b01060bd None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:40.234189 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48855df0-5b54-4f5e-8f30-2542b01060bd None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:40.274907 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0eada6-60c3-4d54-b212-1a8ae33c5e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:40.275521 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0eada6-60c3-4d54-b212-1a8ae33c5e06 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:40.275803 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0eada6-60c3-4d54-b212-1a8ae33c5e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:40.276109 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0eada6-60c3-4d54-b212-1a8ae33c5e06 None admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:40.277000 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0eada6-60c3-4d54-b212-1a8ae33c5e06 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:03:40.277434 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0eada6-60c3-4d54-b212-1a8ae33c5e06 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:40.279045 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0eada6-60c3-4d54-b212-1a8ae33c5e06 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:40.293132 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 604/1231] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 8 01:03:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:03:40.300469 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b3b1a0-e34a-41f5-a7e2-c7aaa00b045b None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:40.377189 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:40.415608 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b3b1a0-e34a-41f5-a7e2-c7aaa00b045b None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:40.452220 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d236bbff-acdf-49c7-9ff3-fa67b5cc8865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:40.452916 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236bbff-acdf-49c7-9ff3-fa67b5cc8865 None admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:40.453208 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236bbff-acdf-49c7-9ff3-fa67b5cc8865 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:40.453540 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236bbff-acdf-49c7-9ff3-fa67b5cc8865 None admin] PATH_INFO: `/v3/projects/78ad2a8fb5264fb4af935e039d28c8de/users/301d30bbd3de4cc69e96bb12729442c6/roles/609d73f56e6a48698397131168abe6df` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:40.470653 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d236bbff-acdf-49c7-9ff3-fa67b5cc8865 None admin] RBAC: Authorizing `identity:create_grant(project_id=78ad2a8fb5264fb4af935e039d28c8de, user_id=301d30bbd3de4cc69e96bb12729442c6, role_id=609d73f56e6a48698397131168abe6df)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:40.472103 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d236bbff-acdf-49c7-9ff3-fa67b5cc8865 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:40.485705 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 628/1232] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 8 01:03:40 2022] PUT /identity/v3/projects/78ad2a8fb5264fb4af935e039d28c8de/users/301d30bbd3de4cc69e96bb12729442c6/roles/609d73f56e6a48698397131168abe6df => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:03:42.867804 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be360a29-78fc-45a8-a1d1-5dd1b966146c None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:42.868373 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be360a29-78fc-45a8-a1d1-5dd1b966146c None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:42.868503 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be360a29-78fc-45a8-a1d1-5dd1b966146c None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:42.870261 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 605/1233] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:03:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:03:42.876420 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889ed19-43d9-446a-a1bc-60d0a50cbc9a None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:42.876874 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889ed19-43d9-446a-a1bc-60d0a50cbc9a None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:42.876996 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889ed19-43d9-446a-a1bc-60d0a50cbc9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:42.896553 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-8889ed19-43d9-446a-a1bc-60d0a50cbc9a None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:03:42.897844 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-8889ed19-43d9-446a-a1bc-60d0a50cbc9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:03:42.972816 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 629/1234] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:03:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 01:03:43.243011 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be360a29-78fc-45a8-a1d1-5dd1b966146c None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:43.269626 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:43.363763 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be360a29-78fc-45a8-a1d1-5dd1b966146c None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:43.410496 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9064b730-001d-44fe-a0a8-59a2acb5a912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:43.411264 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9064b730-001d-44fe-a0a8-59a2acb5a912 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:43.412374 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9064b730-001d-44fe-a0a8-59a2acb5a912 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:43.412650 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9064b730-001d-44fe-a0a8-59a2acb5a912 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:43.425772 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9064b730-001d-44fe-a0a8-59a2acb5a912 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:03:43.431321 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-9064b730-001d-44fe-a0a8-59a2acb5a912 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 01:03:43.431321 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:03:43.431321 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:03:43.431321 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:03:43.431321 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:03:43.431321 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:03:43.431321 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:03:43.431321 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:03:43.431321 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:03:43.431321 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:03:43.431321 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:03:43.431321 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:03:43.431321 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:03:43.431321 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:03:43.431321 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:03:43.431321 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:03:43.432571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:03:43.432571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:03:43.432571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:03:43.432571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:03:43.432571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:03:43.432571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:03:43.432571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:03:43.432571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:03:43.432571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:03:43.432571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:03:43.432571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:03:43.432571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:03:43.432571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:03:43.432571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:03:43.432571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:03:43.432571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:03:43.433757 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:03:43.433757 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:03:43.433757 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:03:43.433757 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:03:43.433757 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:03:43.433757 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:03:43.433757 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:03:43.433757 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:03:43.433757 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:03:43.433757 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:03:43.433757 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 01:03:43.433757 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:03:43.433757 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 606/1235] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Sep 8 01:03:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:03:43.438868 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8889ed19-43d9-446a-a1bc-60d0a50cbc9a None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:43.493560 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:43.556040 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8889ed19-43d9-446a-a1bc-60d0a50cbc9a None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:43.594059 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b031ca-5fd0-442d-8cec-3ab8763efdcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:43.594671 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b031ca-5fd0-442d-8cec-3ab8763efdcc None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:43.595036 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b031ca-5fd0-442d-8cec-3ab8763efdcc None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:43.595382 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b031ca-5fd0-442d-8cec-3ab8763efdcc None admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:43.595974 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b031ca-5fd0-442d-8cec-3ab8763efdcc None admin] RBAC: Adding query filter params (name=service) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:03:43.596306 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b031ca-5fd0-442d-8cec-3ab8763efdcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:43.597698 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b031ca-5fd0-442d-8cec-3ab8763efdcc None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:43.606640 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 630/1236] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Sep 8 01:03:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:03:43.612669 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9064b730-001d-44fe-a0a8-59a2acb5a912 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:43.706006 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:43.723541 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9064b730-001d-44fe-a0a8-59a2acb5a912 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:43.760688 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081de59a-1783-465d-be7f-fb22ec3a3780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:43.761325 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081de59a-1783-465d-be7f-fb22ec3a3780 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:43.761853 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081de59a-1783-465d-be7f-fb22ec3a3780 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:43.762122 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081de59a-1783-465d-be7f-fb22ec3a3780 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:43.773069 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081de59a-1783-465d-be7f-fb22ec3a3780 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:43.774531 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081de59a-1783-465d-be7f-fb22ec3a3780 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:43.790562 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-081de59a-1783-465d-be7f-fb22ec3a3780 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:03:43.790562 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:03:43.790562 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:03:43.790562 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:03:43.790562 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:03:43.790562 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:03:43.790562 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:03:43.790562 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:03:43.790562 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:03:43.790562 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:03:43.790562 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:03:43.790562 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:03:43.790562 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:03:43.790562 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:03:43.790562 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:03:43.790562 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:03:43.791931 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:03:43.791931 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:03:43.791931 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:03:43.791931 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:03:43.791931 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:03:43.791931 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:03:43.791931 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:03:43.791931 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:03:43.791931 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:03:43.791931 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:03:43.791931 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:03:43.791931 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:03:43.791931 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:03:43.791931 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:03:43.791931 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:03:43.791931 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:03:43.791931 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:03:43.793321 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:03:43.793321 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:03:43.793321 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:03:43.793321 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 607/1237] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:03:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:03:43.797941 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b031ca-5fd0-442d-8cec-3ab8763efdcc None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:43.911327 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b031ca-5fd0-442d-8cec-3ab8763efdcc None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:43.922644 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:43.948559 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3794811-39b4-4b39-ab37-2bafb4c98933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:43.949163 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3794811-39b4-4b39-ab37-2bafb4c98933 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:43.949440 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3794811-39b4-4b39-ab37-2bafb4c98933 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:43.949712 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3794811-39b4-4b39-ab37-2bafb4c98933 None admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:43.950353 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3794811-39b4-4b39-ab37-2bafb4c98933 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:03:43.950656 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3794811-39b4-4b39-ab37-2bafb4c98933 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:43.952018 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3794811-39b4-4b39-ab37-2bafb4c98933 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:43.971478 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 631/1238] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:03:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:03:43.972705 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081de59a-1783-465d-be7f-fb22ec3a3780 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:44.083080 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081de59a-1783-465d-be7f-fb22ec3a3780 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:44.120029 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e891adf1-28ff-4c29-8f96-bdd545255ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:44.120735 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e891adf1-28ff-4c29-8f96-bdd545255ec2 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:44.121026 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e891adf1-28ff-4c29-8f96-bdd545255ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:44.121308 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e891adf1-28ff-4c29-8f96-bdd545255ec2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:44.124343 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:44.133994 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e891adf1-28ff-4c29-8f96-bdd545255ec2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:44.135542 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e891adf1-28ff-4c29-8f96-bdd545255ec2 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:44.149033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-e891adf1-28ff-4c29-8f96-bdd545255ec2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 08 01:03:44.149033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:03:44.149033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:03:44.149033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:03:44.149033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:03:44.149033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:03:44.149033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:03:44.149033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:03:44.149033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:03:44.149033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:03:44.149033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:03:44.149033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:03:44.149033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 01:03:44.149033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 01:03:44.149033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 01:03:44.149033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 01:03:44.150231 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:03:44.150231 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:03:44.150231 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 01:03:44.150231 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:03:44.150231 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 01:03:44.150231 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:03:44.150231 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:03:44.150231 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:03:44.150231 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:03:44.150231 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:03:44.150231 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:03:44.150231 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:03:44.150231 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:03:44.150231 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:03:44.150231 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:03:44.150231 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:03:44.150231 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:03:44.150231 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:03:44.151456 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:03:44.151456 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:03:44.151456 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 01:03:44.151456 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 01:03:44.151456 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 01:03:44.151456 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 01:03:44.151456 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 01:03:44.151456 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 01:03:44.151456 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 08 01:03:44.151456 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:03:44.151456 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 608/1239] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:03:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 01:03:44.156093 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3794811-39b4-4b39-ab37-2bafb4c98933 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:44.278096 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3794811-39b4-4b39-ab37-2bafb4c98933 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:44.313819 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7dec25-202d-497b-bb16-c5c9232b5b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:44.314424 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7dec25-202d-497b-bb16-c5c9232b5b4c None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:44.314718 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7dec25-202d-497b-bb16-c5c9232b5b4c None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:44.314970 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7dec25-202d-497b-bb16-c5c9232b5b4c None admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:44.315692 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7dec25-202d-497b-bb16-c5c9232b5b4c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:03:44.315990 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7dec25-202d-497b-bb16-c5c9232b5b4c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:44.317438 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7dec25-202d-497b-bb16-c5c9232b5b4c None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:44.338195 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 632/1240] 10.222.0.42 () {56 vars in 1143 bytes} [Thu Sep 8 01:03:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:03:44.344742 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e891adf1-28ff-4c29-8f96-bdd545255ec2 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:44.459593 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e891adf1-28ff-4c29-8f96-bdd545255ec2 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:44.494245 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad0a89b-06e5-4818-829b-c6d61f2a2364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:44.494862 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad0a89b-06e5-4818-829b-c6d61f2a2364 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:44.495232 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad0a89b-06e5-4818-829b-c6d61f2a2364 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:44.495508 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad0a89b-06e5-4818-829b-c6d61f2a2364 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:44.560214 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:44.568779 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad0a89b-06e5-4818-829b-c6d61f2a2364 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:44.570292 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad0a89b-06e5-4818-829b-c6d61f2a2364 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:44.683609 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 609/1241] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 01:03:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 08 01:03:44.684853 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7dec25-202d-497b-bb16-c5c9232b5b4c None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:44.714202 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:44.796212 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7dec25-202d-497b-bb16-c5c9232b5b4c None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:44.836757 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbe52bb-b12b-488d-8cad-479be69bf450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:44.837598 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe52bb-b12b-488d-8cad-479be69bf450 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:44.837799 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe52bb-b12b-488d-8cad-479be69bf450 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:44.838106 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe52bb-b12b-488d-8cad-479be69bf450 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:44.851869 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbe52bb-b12b-488d-8cad-479be69bf450 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:44.853685 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbe52bb-b12b-488d-8cad-479be69bf450 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:44.870476 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-bcbe52bb-b12b-488d-8cad-479be69bf450 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:03:44.870476 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:03:44.870476 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:03:44.870476 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:03:44.870476 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:03:44.870476 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:03:44.870476 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:03:44.870476 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:03:44.870476 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:03:44.870476 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:03:44.870476 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:03:44.870476 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:03:44.870476 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:03:44.870476 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:03:44.870476 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:03:44.870476 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:03:44.871663 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:03:44.871663 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:03:44.871663 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:03:44.871663 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:03:44.871663 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:03:44.871663 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:03:44.871663 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:03:44.871663 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:03:44.871663 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:03:44.871663 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:03:44.871663 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:03:44.871663 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:03:44.871663 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:03:44.871663 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:03:44.871663 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:03:44.871663 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:03:44.871663 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:03:44.872859 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:03:44.872859 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:03:44.872859 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:03:44.872859 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 633/1242] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:03:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:03:44.877814 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad0a89b-06e5-4818-829b-c6d61f2a2364 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:44.982517 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:44.989547 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad0a89b-06e5-4818-829b-c6d61f2a2364 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:45.029955 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e8f803-85e3-4fcd-9e54-0655c64dc636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:45.030558 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e8f803-85e3-4fcd-9e54-0655c64dc636 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:45.030836 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e8f803-85e3-4fcd-9e54-0655c64dc636 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:45.031100 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e8f803-85e3-4fcd-9e54-0655c64dc636 None admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:45.031731 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e8f803-85e3-4fcd-9e54-0655c64dc636 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:03:45.032037 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e8f803-85e3-4fcd-9e54-0655c64dc636 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:45.033435 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e8f803-85e3-4fcd-9e54-0655c64dc636 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:45.046331 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 610/1243] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:03:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:03:45.052318 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbe52bb-b12b-488d-8cad-479be69bf450 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:45.143627 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:45.167069 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbe52bb-b12b-488d-8cad-479be69bf450 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:45.207950 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a3fc55-abb9-45c0-9755-5071701ccf90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:45.209290 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3fc55-abb9-45c0-9755-5071701ccf90 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:45.209523 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3fc55-abb9-45c0-9755-5071701ccf90 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:45.209785 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3fc55-abb9-45c0-9755-5071701ccf90 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:45.220335 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a3fc55-abb9-45c0-9755-5071701ccf90 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:45.222902 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a3fc55-abb9-45c0-9755-5071701ccf90 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:45.236349 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-78a3fc55-abb9-45c0-9755-5071701ccf90 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:03:45.236349 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:03:45.236349 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:03:45.236349 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:03:45.236349 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:03:45.236349 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:03:45.236349 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:03:45.236349 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:03:45.236349 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:03:45.236349 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:03:45.236349 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:03:45.236349 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:03:45.236349 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 01:03:45.236349 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 01:03:45.236349 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 01:03:45.237286 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 01:03:45.237286 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:03:45.237286 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:03:45.237286 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:03:45.237286 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:03:45.237286 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:03:45.237286 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:03:45.237286 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:03:45.237286 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:03:45.237286 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:03:45.237286 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:03:45.237286 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:03:45.237286 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:03:45.237286 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:03:45.237286 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:03:45.237286 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:03:45.237286 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:03:45.238209 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 01:03:45.238209 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 01:03:45.238209 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 01:03:45.238209 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 01:03:45.238209 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 01:03:45.238209 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 01:03:45.238209 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:03:45.238209 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:03:45.238209 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 634/1244] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 01:03:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:03:45.245384 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e8f803-85e3-4fcd-9e54-0655c64dc636 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:45.360948 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e8f803-85e3-4fcd-9e54-0655c64dc636 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:45.404233 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbb1003-c8b4-41ac-8ba3-a31db9aebba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:45.405276 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb1003-c8b4-41ac-8ba3-a31db9aebba4 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:45.405605 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb1003-c8b4-41ac-8ba3-a31db9aebba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:45.405908 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb1003-c8b4-41ac-8ba3-a31db9aebba4 None admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:45.406744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbb1003-c8b4-41ac-8ba3-a31db9aebba4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:03:45.407044 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbb1003-c8b4-41ac-8ba3-a31db9aebba4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:45.408701 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbb1003-c8b4-41ac-8ba3-a31db9aebba4 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:45.422979 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 611/1245] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 8 01:03:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:03:45.429363 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a3fc55-abb9-45c0-9755-5071701ccf90 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:45.538194 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a3fc55-abb9-45c0-9755-5071701ccf90 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:45.577426 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac486b3f-efa0-4f14-bdc0-f8e4407c1e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:45.578001 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac486b3f-efa0-4f14-bdc0-f8e4407c1e16 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:45.578278 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac486b3f-efa0-4f14-bdc0-f8e4407c1e16 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:45.578551 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac486b3f-efa0-4f14-bdc0-f8e4407c1e16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:45.579273 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac486b3f-efa0-4f14-bdc0-f8e4407c1e16 None admin] RBAC: Adding query filter params (role.id=609d73f56e6a48698397131168abe6df, scope.project.id=78ad2a8fb5264fb4af935e039d28c8de, user.id=301d30bbd3de4cc69e96bb12729442c6) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:03:45.579579 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac486b3f-efa0-4f14-bdc0-f8e4407c1e16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:45.581044 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac486b3f-efa0-4f14-bdc0-f8e4407c1e16 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:45.587463 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 635/1246] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 8 01:03:45 2022] GET /identity/v3/role_assignments?user.id=301d30bbd3de4cc69e96bb12729442c6&scope.project.id=78ad2a8fb5264fb4af935e039d28c8de&role.id=609d73f56e6a48698397131168abe6df => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:03:47.918623 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86fb8e5-93cc-49bc-bb22-6a4f2843339e None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:47.919286 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86fb8e5-93cc-49bc-bb22-6a4f2843339e None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:47.919286 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86fb8e5-93cc-49bc-bb22-6a4f2843339e None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:47.920632 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 612/1247] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:03:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:03:47.925761 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03769e-ad64-4a77-a770-aa0920045920 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:47.926206 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03769e-ad64-4a77-a770-aa0920045920 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:47.926307 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03769e-ad64-4a77-a770-aa0920045920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:47.934022 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:47.946500 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-df03769e-ad64-4a77-a770-aa0920045920 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:03:47.947734 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-df03769e-ad64-4a77-a770-aa0920045920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:03:48.020279 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 636/1248] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:03:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 01:03:48.306661 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86fb8e5-93cc-49bc-bb22-6a4f2843339e None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:48.331667 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:48.434566 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86fb8e5-93cc-49bc-bb22-6a4f2843339e None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:48.471221 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d149f-c13e-4965-b5e3-d6528aa8eb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:48.471872 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d149f-c13e-4965-b5e3-d6528aa8eb0c None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:48.472177 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d149f-c13e-4965-b5e3-d6528aa8eb0c None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:48.472451 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d149f-c13e-4965-b5e3-d6528aa8eb0c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:48.484664 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24d149f-c13e-4965-b5e3-d6528aa8eb0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:03:48.489593 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-a24d149f-c13e-4965-b5e3-d6528aa8eb0c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 01:03:48.489593 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:03:48.489593 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:03:48.489593 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:03:48.489593 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:03:48.489593 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:03:48.489593 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:03:48.489593 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:03:48.489593 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:03:48.489593 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:03:48.489593 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:03:48.489593 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:03:48.489593 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:03:48.489593 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:03:48.489593 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:03:48.489593 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:03:48.490871 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:03:48.490871 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:03:48.490871 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:03:48.490871 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:03:48.490871 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:03:48.490871 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:03:48.490871 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:03:48.490871 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:03:48.490871 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:03:48.490871 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:03:48.490871 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:03:48.490871 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:03:48.490871 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:03:48.490871 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:03:48.490871 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:03:48.490871 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:03:48.491769 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:03:48.491769 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:03:48.491769 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:03:48.491769 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:03:48.491769 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:03:48.491769 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:03:48.491769 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:03:48.491769 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:03:48.491769 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:03:48.491769 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:03:48.491769 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 01:03:48.491769 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:03:48.491769 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 613/1249] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 8 01:03:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:03:48.497201 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df03769e-ad64-4a77-a770-aa0920045920 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:48.601967 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df03769e-ad64-4a77-a770-aa0920045920 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:48.635126 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:48.640996 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224baa16-366e-464a-b4f6-4acc6b1d8a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:48.641616 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224baa16-366e-464a-b4f6-4acc6b1d8a48 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:48.641889 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224baa16-366e-464a-b4f6-4acc6b1d8a48 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:48.642158 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224baa16-366e-464a-b4f6-4acc6b1d8a48 None admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:48.642829 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224baa16-366e-464a-b4f6-4acc6b1d8a48 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:03:48.643125 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224baa16-366e-464a-b4f6-4acc6b1d8a48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:48.644569 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224baa16-366e-464a-b4f6-4acc6b1d8a48 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:48.659177 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 637/1250] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 01:03:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:03:48.660438 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d149f-c13e-4965-b5e3-d6528aa8eb0c None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:48.743756 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:48.768406 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d149f-c13e-4965-b5e3-d6528aa8eb0c None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:48.805118 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f27148-aa2a-4803-a5aa-6e975536e882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:48.805762 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f27148-aa2a-4803-a5aa-6e975536e882 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:48.806346 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f27148-aa2a-4803-a5aa-6e975536e882 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:48.806618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f27148-aa2a-4803-a5aa-6e975536e882 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:48.816794 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f27148-aa2a-4803-a5aa-6e975536e882 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:48.819040 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f27148-aa2a-4803-a5aa-6e975536e882 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:48.833165 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-c0f27148-aa2a-4803-a5aa-6e975536e882 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:03:48.833165 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:03:48.833165 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:03:48.833165 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:03:48.833165 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:03:48.833165 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:03:48.833165 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:03:48.833165 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:03:48.833165 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:03:48.833165 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:03:48.833165 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:03:48.833165 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:03:48.833165 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:03:48.833165 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:03:48.833165 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:03:48.833165 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:03:48.835179 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:03:48.835179 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:03:48.835179 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:03:48.835179 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:03:48.835179 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:03:48.835179 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:03:48.835179 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:03:48.835179 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:03:48.835179 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:03:48.835179 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:03:48.835179 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:03:48.835179 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:03:48.835179 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:03:48.835179 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:03:48.835179 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:03:48.835179 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:03:48.835179 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:03:48.836504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:03:48.836504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:03:48.836504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:03:48.836504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 614/1251] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:03:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:03:48.842593 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224baa16-366e-464a-b4f6-4acc6b1d8a48 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:48.947438 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224baa16-366e-464a-b4f6-4acc6b1d8a48 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:48.980245 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e1e0d3-8736-481e-8ef3-8a3196870949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:48.980848 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e1e0d3-8736-481e-8ef3-8a3196870949 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:48.981120 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e1e0d3-8736-481e-8ef3-8a3196870949 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:48.981385 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e1e0d3-8736-481e-8ef3-8a3196870949 None admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:48.982020 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e1e0d3-8736-481e-8ef3-8a3196870949 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:03:48.982320 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e1e0d3-8736-481e-8ef3-8a3196870949 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:48.983670 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e1e0d3-8736-481e-8ef3-8a3196870949 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:48.995997 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 638/1252] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:03:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:03:49.001831 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f27148-aa2a-4803-a5aa-6e975536e882 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:49.103954 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f27148-aa2a-4803-a5aa-6e975536e882 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:49.129499 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:49.138644 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db03cbb4-92f6-4287-ae04-1e4ba2f9dd00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:49.139268 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db03cbb4-92f6-4287-ae04-1e4ba2f9dd00 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:49.139547 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db03cbb4-92f6-4287-ae04-1e4ba2f9dd00 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:49.139819 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db03cbb4-92f6-4287-ae04-1e4ba2f9dd00 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:49.147770 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db03cbb4-92f6-4287-ae04-1e4ba2f9dd00 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:49.149256 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db03cbb4-92f6-4287-ae04-1e4ba2f9dd00 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:49.160023 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-db03cbb4-92f6-4287-ae04-1e4ba2f9dd00 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 08 01:03:49.160023 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:03:49.160023 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:03:49.160023 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:03:49.160023 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:03:49.160023 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:03:49.160023 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:03:49.160023 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:03:49.160023 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:03:49.160023 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:03:49.160023 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:03:49.160023 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:03:49.160023 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 01:03:49.160023 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 01:03:49.160023 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 01:03:49.160023 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 01:03:49.161175 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:03:49.161175 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:03:49.161175 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 01:03:49.161175 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:03:49.161175 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 01:03:49.161175 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:03:49.161175 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:03:49.161175 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:03:49.161175 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:03:49.161175 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:03:49.161175 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:03:49.161175 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:03:49.161175 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:03:49.161175 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:03:49.161175 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:03:49.161175 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:03:49.161175 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:03:49.161175 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:03:49.162150 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:03:49.162150 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:03:49.162150 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 01:03:49.162150 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 01:03:49.162150 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 01:03:49.162150 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 01:03:49.162150 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 01:03:49.162150 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 01:03:49.162150 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 08 01:03:49.162150 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:03:49.162150 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 615/1253] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:03:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:03:49.167273 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e1e0d3-8736-481e-8ef3-8a3196870949 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:49.225981 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:49.271572 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e1e0d3-8736-481e-8ef3-8a3196870949 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:49.305357 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42a99e2-0648-4847-ab33-16382a443a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:49.305959 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a99e2-0648-4847-ab33-16382a443a23 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:49.306234 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a99e2-0648-4847-ab33-16382a443a23 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:49.306504 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a99e2-0648-4847-ab33-16382a443a23 None admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:49.307228 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42a99e2-0648-4847-ab33-16382a443a23 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:03:49.307535 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42a99e2-0648-4847-ab33-16382a443a23 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:49.308989 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42a99e2-0648-4847-ab33-16382a443a23 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:49.327183 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 639/1254] 10.222.0.42 () {56 vars in 1143 bytes} [Thu Sep 8 01:03:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:03:49.332950 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db03cbb4-92f6-4287-ae04-1e4ba2f9dd00 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:49.428868 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db03cbb4-92f6-4287-ae04-1e4ba2f9dd00 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:49.461197 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6e0c87-7ff0-4b52-900d-22ed88f14ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:49.461777 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6e0c87-7ff0-4b52-900d-22ed88f14ddb None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:49.462052 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6e0c87-7ff0-4b52-900d-22ed88f14ddb None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:49.462319 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6e0c87-7ff0-4b52-900d-22ed88f14ddb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:49.515530 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:49.527842 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6e0c87-7ff0-4b52-900d-22ed88f14ddb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:49.529312 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6e0c87-7ff0-4b52-900d-22ed88f14ddb None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:49.622763 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 616/1255] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 01:03:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 01:03:49.628491 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42a99e2-0648-4847-ab33-16382a443a23 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:49.725585 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42a99e2-0648-4847-ab33-16382a443a23 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:49.738056 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:49.758831 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccadc803-5b32-4b88-b5fc-092ef8657ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:49.759438 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccadc803-5b32-4b88-b5fc-092ef8657ba1 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:49.759709 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccadc803-5b32-4b88-b5fc-092ef8657ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:49.759972 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccadc803-5b32-4b88-b5fc-092ef8657ba1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:49.769818 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccadc803-5b32-4b88-b5fc-092ef8657ba1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:49.772083 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccadc803-5b32-4b88-b5fc-092ef8657ba1 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:49.785849 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-ccadc803-5b32-4b88-b5fc-092ef8657ba1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:03:49.785849 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:03:49.785849 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:03:49.785849 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:03:49.785849 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:03:49.785849 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:03:49.785849 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:03:49.785849 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:03:49.785849 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:03:49.785849 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:03:49.785849 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:03:49.785849 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:03:49.785849 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:03:49.785849 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:03:49.785849 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:03:49.785849 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:03:49.787109 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:03:49.787109 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:03:49.787109 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:03:49.787109 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:03:49.787109 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:03:49.787109 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:03:49.787109 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:03:49.787109 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:03:49.787109 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:03:49.787109 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:03:49.787109 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:03:49.787109 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:03:49.787109 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:03:49.787109 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:03:49.787109 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:03:49.787109 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:03:49.787109 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:03:49.788352 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:03:49.788352 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:03:49.788352 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:03:49.788352 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 640/1256] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:03:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:03:49.792935 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6e0c87-7ff0-4b52-900d-22ed88f14ddb None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:49.877703 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:49.890764 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6e0c87-7ff0-4b52-900d-22ed88f14ddb None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:49.923260 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86453f9b-66d3-46a3-8f04-ccde964c939a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:49.923865 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86453f9b-66d3-46a3-8f04-ccde964c939a None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:49.924159 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86453f9b-66d3-46a3-8f04-ccde964c939a None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:49.924442 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86453f9b-66d3-46a3-8f04-ccde964c939a None admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:49.925077 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86453f9b-66d3-46a3-8f04-ccde964c939a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:03:49.925373 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86453f9b-66d3-46a3-8f04-ccde964c939a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:49.926736 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86453f9b-66d3-46a3-8f04-ccde964c939a None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:49.938143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 617/1257] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:03:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:03:49.944380 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccadc803-5b32-4b88-b5fc-092ef8657ba1 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:50.040533 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccadc803-5b32-4b88-b5fc-092ef8657ba1 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:50.072396 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66d3946-8057-4e18-b7a7-6289614ef632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:50.073036 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d3946-8057-4e18-b7a7-6289614ef632 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:50.073311 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d3946-8057-4e18-b7a7-6289614ef632 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:50.073581 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d3946-8057-4e18-b7a7-6289614ef632 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:50.079870 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66d3946-8057-4e18-b7a7-6289614ef632 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:50.081319 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66d3946-8057-4e18-b7a7-6289614ef632 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:50.092338 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-d66d3946-8057-4e18-b7a7-6289614ef632 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:03:50.092338 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:03:50.092338 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:03:50.092338 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:03:50.092338 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:03:50.092338 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:03:50.092338 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:03:50.092338 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:03:50.092338 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:03:50.092338 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:03:50.092338 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:03:50.092338 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:03:50.092338 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 01:03:50.092338 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 01:03:50.092338 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 01:03:50.093356 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 01:03:50.093356 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:03:50.093356 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:03:50.093356 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:03:50.093356 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:03:50.093356 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:03:50.093356 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:03:50.093356 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:03:50.093356 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:03:50.093356 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:03:50.093356 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:03:50.093356 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:03:50.093356 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:03:50.093356 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:03:50.093356 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:03:50.093356 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:03:50.093356 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:03:50.094286 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 01:03:50.094286 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 01:03:50.094286 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 01:03:50.094286 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 01:03:50.094286 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 01:03:50.094286 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 01:03:50.094286 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:03:50.094286 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:03:50.094286 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 641/1258] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 01:03:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 01:03:50.099663 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86453f9b-66d3-46a3-8f04-ccde964c939a None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:50.203750 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86453f9b-66d3-46a3-8f04-ccde964c939a None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:50.241753 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc5077c-28a4-4d5d-adb6-477100196cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:50.242360 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5077c-28a4-4d5d-adb6-477100196cbe None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:50.242642 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5077c-28a4-4d5d-adb6-477100196cbe None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:50.242919 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5077c-28a4-4d5d-adb6-477100196cbe None admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:50.243603 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc5077c-28a4-4d5d-adb6-477100196cbe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:03:50.243909 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc5077c-28a4-4d5d-adb6-477100196cbe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:50.245382 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc5077c-28a4-4d5d-adb6-477100196cbe None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:50.257481 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 618/1259] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 8 01:03:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:03:50.263785 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66d3946-8057-4e18-b7a7-6289614ef632 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:50.289304 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:50.361151 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66d3946-8057-4e18-b7a7-6289614ef632 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:50.394678 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0670629-2052-4afb-9a71-f35594a9d066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:50.395258 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0670629-2052-4afb-9a71-f35594a9d066 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:50.395536 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0670629-2052-4afb-9a71-f35594a9d066 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:50.395847 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0670629-2052-4afb-9a71-f35594a9d066 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:50.396769 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0670629-2052-4afb-9a71-f35594a9d066 None admin] RBAC: Adding query filter params (role.id=34de369ec7c54715aa83a9052a26c3f2, scope.project.id=78ad2a8fb5264fb4af935e039d28c8de, user.id=301d30bbd3de4cc69e96bb12729442c6) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:03:50.397072 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0670629-2052-4afb-9a71-f35594a9d066 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:50.398485 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0670629-2052-4afb-9a71-f35594a9d066 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:50.403655 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 642/1260] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 8 01:03:50 2022] GET /identity/v3/role_assignments?user.id=301d30bbd3de4cc69e96bb12729442c6&scope.project.id=78ad2a8fb5264fb4af935e039d28c8de&role.id=34de369ec7c54715aa83a9052a26c3f2 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:03:52.508256 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a26c22-cebb-42f1-b7a7-a1bcc1bd2630 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:52.508782 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a26c22-cebb-42f1-b7a7-a1bcc1bd2630 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:52.508782 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a26c22-cebb-42f1-b7a7-a1bcc1bd2630 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:52.510113 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 619/1261] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:03:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:03:52.515554 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5152e6-733c-4bd4-a397-d8488ff37ff3 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:52.516011 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5152e6-733c-4bd4-a397-d8488ff37ff3 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:52.516105 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5152e6-733c-4bd4-a397-d8488ff37ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:52.533984 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-7f5152e6-733c-4bd4-a397-d8488ff37ff3 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:03:52.535146 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-7f5152e6-733c-4bd4-a397-d8488ff37ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:03:52.599735 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 643/1262] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:03:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 01:03:52.869016 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a26c22-cebb-42f1-b7a7-a1bcc1bd2630 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:52.879676 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:52.968857 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a26c22-cebb-42f1-b7a7-a1bcc1bd2630 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:53.003279 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4da5f79-14e9-4820-a953-1425d717c57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:53.003909 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4da5f79-14e9-4820-a953-1425d717c57f None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:53.004219 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4da5f79-14e9-4820-a953-1425d717c57f None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:53.004498 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4da5f79-14e9-4820-a953-1425d717c57f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:53.015415 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4da5f79-14e9-4820-a953-1425d717c57f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:03:53.020127 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-f4da5f79-14e9-4820-a953-1425d717c57f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 01:03:53.020127 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:03:53.020127 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:03:53.020127 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:03:53.020127 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:03:53.020127 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:03:53.020127 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:03:53.020127 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:03:53.020127 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:03:53.020127 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:03:53.020127 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:03:53.020127 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:03:53.020127 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:03:53.020127 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:03:53.020127 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:03:53.020127 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:03:53.021480 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:03:53.021480 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:03:53.021480 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:03:53.021480 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:03:53.021480 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:03:53.021480 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:03:53.021480 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:03:53.021480 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:03:53.021480 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:03:53.021480 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:03:53.021480 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:03:53.021480 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:03:53.021480 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:03:53.021480 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:03:53.021480 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:03:53.021480 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:03:53.022357 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:03:53.022357 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:03:53.022357 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:03:53.022357 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:03:53.022357 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:03:53.022357 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:03:53.022357 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:03:53.022357 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:03:53.022357 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:03:53.022357 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:03:53.022357 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 01:03:53.022357 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:03:53.022357 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 620/1263] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 8 01:03:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:03:53.027591 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5152e6-733c-4bd4-a397-d8488ff37ff3 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:53.046079 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:53.125101 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5152e6-733c-4bd4-a397-d8488ff37ff3 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:53.158103 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e6fb42-6102-4b70-809b-4c4463259412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:53.158691 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e6fb42-6102-4b70-809b-4c4463259412 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:53.158967 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e6fb42-6102-4b70-809b-4c4463259412 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:53.159237 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e6fb42-6102-4b70-809b-4c4463259412 None admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:53.159888 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e6fb42-6102-4b70-809b-4c4463259412 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:03:53.160215 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e6fb42-6102-4b70-809b-4c4463259412 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:53.162087 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e6fb42-6102-4b70-809b-4c4463259412 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:53.170127 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 644/1264] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 01:03:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:03:53.176204 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4da5f79-14e9-4820-a953-1425d717c57f None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:53.247999 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:53.280989 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4da5f79-14e9-4820-a953-1425d717c57f None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:53.314483 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3136ec42-1225-4ff0-81fb-34f5a008ca88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:53.315102 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3136ec42-1225-4ff0-81fb-34f5a008ca88 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:53.315379 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3136ec42-1225-4ff0-81fb-34f5a008ca88 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:53.315650 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3136ec42-1225-4ff0-81fb-34f5a008ca88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:53.325690 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3136ec42-1225-4ff0-81fb-34f5a008ca88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:53.327131 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3136ec42-1225-4ff0-81fb-34f5a008ca88 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:53.340741 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-3136ec42-1225-4ff0-81fb-34f5a008ca88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:03:53.340741 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:03:53.340741 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:03:53.340741 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:03:53.340741 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:03:53.340741 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:03:53.340741 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:03:53.340741 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:03:53.340741 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:03:53.340741 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:03:53.340741 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:03:53.340741 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:03:53.340741 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:03:53.340741 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:03:53.340741 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:03:53.340741 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:03:53.346561 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:03:53.346561 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:03:53.346561 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:03:53.346561 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:03:53.346561 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:03:53.346561 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:03:53.346561 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:03:53.346561 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:03:53.346561 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:03:53.346561 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:03:53.346561 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:03:53.346561 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:03:53.346561 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:03:53.346561 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:03:53.346561 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:03:53.346561 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:03:53.346561 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:03:53.347733 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:03:53.347733 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:03:53.347733 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:03:53.347733 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 621/1265] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:03:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:03:53.348016 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e6fb42-6102-4b70-809b-4c4463259412 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:53.427068 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:53.445974 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e6fb42-6102-4b70-809b-4c4463259412 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:53.479055 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a83326c-87cc-46c7-a50e-1eefb3489943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:53.479646 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a83326c-87cc-46c7-a50e-1eefb3489943 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:53.479925 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a83326c-87cc-46c7-a50e-1eefb3489943 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:53.480225 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a83326c-87cc-46c7-a50e-1eefb3489943 None admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:53.480862 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a83326c-87cc-46c7-a50e-1eefb3489943 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:03:53.481165 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a83326c-87cc-46c7-a50e-1eefb3489943 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:53.482531 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a83326c-87cc-46c7-a50e-1eefb3489943 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:53.493914 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 645/1266] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:03:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:03:53.499746 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3136ec42-1225-4ff0-81fb-34f5a008ca88 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:53.597312 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3136ec42-1225-4ff0-81fb-34f5a008ca88 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:53.616817 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:53.631304 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119615eb-baea-40ed-91ae-871fa743ed92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:53.631934 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119615eb-baea-40ed-91ae-871fa743ed92 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:53.632295 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119615eb-baea-40ed-91ae-871fa743ed92 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:53.632918 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119615eb-baea-40ed-91ae-871fa743ed92 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:53.640326 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119615eb-baea-40ed-91ae-871fa743ed92 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:53.641770 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119615eb-baea-40ed-91ae-871fa743ed92 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:53.653303 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-119615eb-baea-40ed-91ae-871fa743ed92 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 08 01:03:53.653303 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:03:53.653303 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:03:53.653303 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:03:53.653303 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:03:53.653303 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:03:53.653303 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:03:53.653303 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:03:53.653303 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:03:53.653303 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:03:53.653303 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:03:53.653303 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:03:53.653303 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 01:03:53.653303 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 01:03:53.653303 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 01:03:53.653303 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 01:03:53.654378 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:03:53.654378 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:03:53.654378 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 01:03:53.654378 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:03:53.654378 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 01:03:53.654378 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:03:53.654378 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:03:53.654378 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:03:53.654378 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:03:53.654378 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:03:53.654378 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:03:53.654378 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:03:53.654378 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:03:53.654378 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:03:53.654378 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:03:53.654378 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:03:53.654378 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:03:53.654378 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:03:53.655709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:03:53.655709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:03:53.655709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 01:03:53.655709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 01:03:53.655709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 01:03:53.655709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 01:03:53.655709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 01:03:53.655709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 01:03:53.655709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 08 01:03:53.655709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:03:53.655709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 622/1267] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:03:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:03:53.660246 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a83326c-87cc-46c7-a50e-1eefb3489943 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:53.757550 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a83326c-87cc-46c7-a50e-1eefb3489943 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:53.791175 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ce8f67-e95e-43b7-97d8-426ed14d4940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:53.791758 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce8f67-e95e-43b7-97d8-426ed14d4940 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:53.792029 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce8f67-e95e-43b7-97d8-426ed14d4940 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:53.792325 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce8f67-e95e-43b7-97d8-426ed14d4940 None admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:53.793037 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ce8f67-e95e-43b7-97d8-426ed14d4940 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:03:53.793340 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ce8f67-e95e-43b7-97d8-426ed14d4940 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:53.794745 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ce8f67-e95e-43b7-97d8-426ed14d4940 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:53.803336 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:53.811556 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 646/1268] 10.222.0.42 () {56 vars in 1143 bytes} [Thu Sep 8 01:03:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:03:53.817438 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119615eb-baea-40ed-91ae-871fa743ed92 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:53.916586 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119615eb-baea-40ed-91ae-871fa743ed92 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:53.949088 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdcbd15-ba97-493e-9642-b0eb2f45e084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:53.949702 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdcbd15-ba97-493e-9642-b0eb2f45e084 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:53.949980 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdcbd15-ba97-493e-9642-b0eb2f45e084 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:53.950253 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdcbd15-ba97-493e-9642-b0eb2f45e084 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:53.961043 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdcbd15-ba97-493e-9642-b0eb2f45e084 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:53.962501 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdcbd15-ba97-493e-9642-b0eb2f45e084 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:53.976468 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-0bdcbd15-ba97-493e-9642-b0eb2f45e084 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:03:53.976468 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:03:53.976468 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:03:53.976468 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:03:53.976468 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:03:53.976468 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:03:53.976468 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:03:53.976468 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:03:53.976468 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:03:53.976468 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:03:53.976468 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:03:53.976468 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:03:53.976468 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:03:53.976468 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:03:53.976468 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:03:53.976468 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:03:53.977435 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:03:53.977435 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:03:53.977435 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:03:53.977435 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:03:53.977435 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:03:53.977435 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:03:53.977435 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:03:53.977435 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:03:53.977435 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:03:53.977435 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:03:53.977435 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:03:53.977435 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:03:53.977435 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:03:53.977435 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:03:53.977435 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:03:53.977435 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:03:53.977435 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:03:53.978431 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:03:53.978431 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:03:53.978431 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:03:53.978431 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 623/1269] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:03:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:03:53.983344 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ce8f67-e95e-43b7-97d8-426ed14d4940 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:54.079591 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ce8f67-e95e-43b7-97d8-426ed14d4940 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:54.112260 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a9d892-73a1-4286-ad49-1c30b06268d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:54.112849 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a9d892-73a1-4286-ad49-1c30b06268d9 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:54.113126 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a9d892-73a1-4286-ad49-1c30b06268d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:54.113396 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a9d892-73a1-4286-ad49-1c30b06268d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:54.114028 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a9d892-73a1-4286-ad49-1c30b06268d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:03:54.114330 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a9d892-73a1-4286-ad49-1c30b06268d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:54.116380 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a9d892-73a1-4286-ad49-1c30b06268d9 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:54.127161 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 647/1270] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:03:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:03:54.132959 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdcbd15-ba97-493e-9642-b0eb2f45e084 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:54.164688 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:54.242874 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdcbd15-ba97-493e-9642-b0eb2f45e084 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:54.275887 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd9e4e6-3f07-4aa3-9f06-d996b8450cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:54.276528 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9e4e6-3f07-4aa3-9f06-d996b8450cba None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:54.276814 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9e4e6-3f07-4aa3-9f06-d996b8450cba None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:54.277088 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9e4e6-3f07-4aa3-9f06-d996b8450cba None admin] PATH_INFO: `/v3/projects/service` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:54.283461 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd9e4e6-3f07-4aa3-9f06-d996b8450cba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:54.284933 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd9e4e6-3f07-4aa3-9f06-d996b8450cba None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:54.297282 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-1cd9e4e6-3f07-4aa3-9f06-d996b8450cba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:03:54.297282 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:03:54.297282 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:03:54.297282 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:03:54.297282 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:03:54.297282 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:03:54.297282 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:03:54.297282 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:03:54.297282 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:03:54.297282 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:03:54.297282 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:03:54.297282 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:03:54.297282 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 01:03:54.297282 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 01:03:54.297282 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 01:03:54.298359 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 01:03:54.298359 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:03:54.298359 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:03:54.298359 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:03:54.298359 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:03:54.298359 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:03:54.298359 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:03:54.298359 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:03:54.298359 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:03:54.298359 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:03:54.298359 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:03:54.298359 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:03:54.298359 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:03:54.298359 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:03:54.298359 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:03:54.298359 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:03:54.298359 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:03:54.299250 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 01:03:54.299250 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 01:03:54.299250 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 01:03:54.299250 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 01:03:54.299250 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 01:03:54.299250 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 01:03:54.299250 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:03:54.299250 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:03:54.299250 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 624/1271] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 01:03:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:03:54.304527 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a9d892-73a1-4286-ad49-1c30b06268d9 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:54.362431 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:54.404008 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a9d892-73a1-4286-ad49-1c30b06268d9 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:54.437123 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953ee9ac-53b1-45ae-a98b-d58d78b36ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:54.437746 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ee9ac-53b1-45ae-a98b-d58d78b36ff6 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:54.438023 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ee9ac-53b1-45ae-a98b-d58d78b36ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:54.438292 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ee9ac-53b1-45ae-a98b-d58d78b36ff6 None admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:54.438974 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953ee9ac-53b1-45ae-a98b-d58d78b36ff6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:03:54.439276 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953ee9ac-53b1-45ae-a98b-d58d78b36ff6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:54.440750 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953ee9ac-53b1-45ae-a98b-d58d78b36ff6 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:54.452405 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 648/1272] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 8 01:03:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:03:54.459102 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd9e4e6-3f07-4aa3-9f06-d996b8450cba None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:54.548348 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:54.557056 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd9e4e6-3f07-4aa3-9f06-d996b8450cba None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:54.590170 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed26fd6-cae2-4810-bb7d-0b17a19e2804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:54.590809 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed26fd6-cae2-4810-bb7d-0b17a19e2804 None admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:54.591092 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed26fd6-cae2-4810-bb7d-0b17a19e2804 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:54.591365 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed26fd6-cae2-4810-bb7d-0b17a19e2804 None admin] PATH_INFO: `/v3/projects/78ad2a8fb5264fb4af935e039d28c8de/users/301d30bbd3de4cc69e96bb12729442c6/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:54.593257 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed26fd6-cae2-4810-bb7d-0b17a19e2804 None admin] RBAC: Authorizing `identity:create_grant(project_id=78ad2a8fb5264fb4af935e039d28c8de, user_id=301d30bbd3de4cc69e96bb12729442c6, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:54.594690 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed26fd6-cae2-4810-bb7d-0b17a19e2804 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:54.606497 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 625/1273] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Sep 8 01:03:54 2022] PUT /identity/v3/projects/78ad2a8fb5264fb4af935e039d28c8de/users/301d30bbd3de4cc69e96bb12729442c6/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:03:56.730649 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88acfe8-9bd0-46ff-92ac-8189a10ed1da None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:56.731168 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88acfe8-9bd0-46ff-92ac-8189a10ed1da None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:56.731168 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88acfe8-9bd0-46ff-92ac-8189a10ed1da None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:56.732573 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 649/1274] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:03:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:03:56.737760 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def5677-3815-487b-a7df-6221542af7fc None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:56.738164 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def5677-3815-487b-a7df-6221542af7fc None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:56.738306 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def5677-3815-487b-a7df-6221542af7fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:56.757922 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-0def5677-3815-487b-a7df-6221542af7fc None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:03:56.759116 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-0def5677-3815-487b-a7df-6221542af7fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:03:56.824473 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 626/1275] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:03:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 01:03:57.094173 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88acfe8-9bd0-46ff-92ac-8189a10ed1da None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:57.195085 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88acfe8-9bd0-46ff-92ac-8189a10ed1da None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:57.213199 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:57.234599 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3ad6d2-c441-4ead-8f9a-a102358610ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:57.235465 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ad6d2-c441-4ead-8f9a-a102358610ba None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:57.235657 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ad6d2-c441-4ead-8f9a-a102358610ba None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:57.235989 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ad6d2-c441-4ead-8f9a-a102358610ba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:57.246600 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3ad6d2-c441-4ead-8f9a-a102358610ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 01:03:57.251347 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-4c3ad6d2-c441-4ead-8f9a-a102358610ba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 01:03:57.251347 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:03:57.251347 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:03:57.251347 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:03:57.251347 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:03:57.251347 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:03:57.251347 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:03:57.251347 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:03:57.251347 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:03:57.251347 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:03:57.251347 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:03:57.251347 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:03:57.251347 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 01:03:57.251347 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 01:03:57.251347 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 01:03:57.251347 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 01:03:57.252448 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 01:03:57.252448 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 01:03:57.252448 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 01:03:57.252448 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 01:03:57.252448 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:03:57.252448 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:03:57.252448 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:03:57.252448 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:03:57.252448 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:03:57.252448 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:03:57.252448 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:03:57.252448 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:03:57.252448 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:03:57.252448 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:03:57.252448 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:03:57.252448 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:03:57.253365 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:03:57.253365 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:03:57.253365 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:03:57.253365 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:03:57.253365 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 01:03:57.253365 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 01:03:57.253365 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 01:03:57.253365 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 01:03:57.253365 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 01:03:57.253365 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 01:03:57.253365 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 01:03:57.253365 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:03:57.253365 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 650/1276] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Sep 8 01:03:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:03:57.259215 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0def5677-3815-487b-a7df-6221542af7fc None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:57.337686 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:57.356620 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0def5677-3815-487b-a7df-6221542af7fc None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:57.391078 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff3433-2ee3-459f-9eb3-025b95f7de5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:57.391672 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff3433-2ee3-459f-9eb3-025b95f7de5b None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:57.391954 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff3433-2ee3-459f-9eb3-025b95f7de5b None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:57.392258 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff3433-2ee3-459f-9eb3-025b95f7de5b None admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:57.392919 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ff3433-2ee3-459f-9eb3-025b95f7de5b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:03:57.393225 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ff3433-2ee3-459f-9eb3-025b95f7de5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:57.394595 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ff3433-2ee3-459f-9eb3-025b95f7de5b None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:57.402445 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 627/1277] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Sep 8 01:03:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:03:57.408239 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3ad6d2-c441-4ead-8f9a-a102358610ba None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:57.506001 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3ad6d2-c441-4ead-8f9a-a102358610ba None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:57.539146 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825c4625-1b6e-429c-a535-94d17eaa1f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:57.539759 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c4625-1b6e-429c-a535-94d17eaa1f8a None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:57.540038 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c4625-1b6e-429c-a535-94d17eaa1f8a None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:57.540344 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c4625-1b6e-429c-a535-94d17eaa1f8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:57.550044 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825c4625-1b6e-429c-a535-94d17eaa1f8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:57.551497 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825c4625-1b6e-429c-a535-94d17eaa1f8a None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:57.566144 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-825c4625-1b6e-429c-a535-94d17eaa1f8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:03:57.566144 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:03:57.566144 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:03:57.566144 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:03:57.566144 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:03:57.566144 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:03:57.566144 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:03:57.566144 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:03:57.566144 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:03:57.566144 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:03:57.566144 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:03:57.566144 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:03:57.566144 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:03:57.566144 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:03:57.566144 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:03:57.566144 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:03:57.567149 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:03:57.567149 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:03:57.567149 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:03:57.567149 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:03:57.567149 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:03:57.567149 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:03:57.567149 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:03:57.567149 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:03:57.567149 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:03:57.567149 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:03:57.567149 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:03:57.567149 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:03:57.567149 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:03:57.567149 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:03:57.567149 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:03:57.567149 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:03:57.567149 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:03:57.568382 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:03:57.568382 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:03:57.568382 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:03:57.568382 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 651/1278] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:03:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:03:57.572987 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff3433-2ee3-459f-9eb3-025b95f7de5b None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:57.670641 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff3433-2ee3-459f-9eb3-025b95f7de5b None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:57.703236 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27f3e81-87f6-4708-8482-8880065fc82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:57.703835 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27f3e81-87f6-4708-8482-8880065fc82a None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:57.704118 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27f3e81-87f6-4708-8482-8880065fc82a None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:57.704423 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27f3e81-87f6-4708-8482-8880065fc82a None admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:57.705060 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27f3e81-87f6-4708-8482-8880065fc82a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:03:57.705364 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27f3e81-87f6-4708-8482-8880065fc82a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:57.706735 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27f3e81-87f6-4708-8482-8880065fc82a None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:57.718103 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 628/1279] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:03:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:03:57.724417 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825c4625-1b6e-429c-a535-94d17eaa1f8a None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:57.754331 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:57.820978 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825c4625-1b6e-429c-a535-94d17eaa1f8a None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:57.853151 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06857c5b-85d9-403a-a926-16dfddc0da2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:57.853758 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06857c5b-85d9-403a-a926-16dfddc0da2f None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:57.854031 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06857c5b-85d9-403a-a926-16dfddc0da2f None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:57.854297 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06857c5b-85d9-403a-a926-16dfddc0da2f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:57.861554 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06857c5b-85d9-403a-a926-16dfddc0da2f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:57.862964 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06857c5b-85d9-403a-a926-16dfddc0da2f None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:57.874396 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-06857c5b-85d9-403a-a926-16dfddc0da2f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 08 01:03:57.874396 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:03:57.874396 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:03:57.874396 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:03:57.874396 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:03:57.874396 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:03:57.874396 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:03:57.874396 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:03:57.874396 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:03:57.874396 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:03:57.874396 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:03:57.874396 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:03:57.874396 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 01:03:57.874396 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 01:03:57.874396 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 01:03:57.874396 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 01:03:57.875490 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:03:57.875490 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:03:57.875490 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 01:03:57.875490 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:03:57.875490 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 01:03:57.875490 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 01:03:57.875490 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:03:57.875490 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:03:57.875490 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:03:57.875490 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:03:57.875490 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:03:57.875490 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:03:57.875490 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:03:57.875490 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:03:57.875490 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:03:57.875490 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:03:57.875490 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:03:57.875490 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:03:57.876583 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:03:57.876583 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:03:57.876583 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 01:03:57.876583 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 01:03:57.876583 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 01:03:57.876583 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 01:03:57.876583 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 01:03:57.876583 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 01:03:57.876583 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 08 01:03:57.876583 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:03:57.876583 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 652/1280] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:03:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:03:57.881461 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27f3e81-87f6-4708-8482-8880065fc82a None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:57.885270 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:57.980154 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27f3e81-87f6-4708-8482-8880065fc82a None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:58.012451 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6431bcd-fd0c-4fbb-8be8-3291942543d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:58.013048 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6431bcd-fd0c-4fbb-8be8-3291942543d1 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:58.013331 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6431bcd-fd0c-4fbb-8be8-3291942543d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:58.013604 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6431bcd-fd0c-4fbb-8be8-3291942543d1 None admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:58.014324 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6431bcd-fd0c-4fbb-8be8-3291942543d1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:03:58.014722 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6431bcd-fd0c-4fbb-8be8-3291942543d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:58.016183 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6431bcd-fd0c-4fbb-8be8-3291942543d1 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:58.033663 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 629/1281] 10.222.0.42 () {56 vars in 1143 bytes} [Thu Sep 8 01:03:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:03:58.040160 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06857c5b-85d9-403a-a926-16dfddc0da2f None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:58.126256 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:58.136217 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06857c5b-85d9-403a-a926-16dfddc0da2f None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:58.169363 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34239673-3b3e-40d0-8ea2-ab9bbf1a28ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:58.169950 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34239673-3b3e-40d0-8ea2-ab9bbf1a28ac None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:58.170224 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34239673-3b3e-40d0-8ea2-ab9bbf1a28ac None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:58.170489 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34239673-3b3e-40d0-8ea2-ab9bbf1a28ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:58.235075 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34239673-3b3e-40d0-8ea2-ab9bbf1a28ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:58.236569 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34239673-3b3e-40d0-8ea2-ab9bbf1a28ac None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:58.321994 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:58.332021 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 653/1282] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Sep 8 01:03:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 01:03:58.338183 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6431bcd-fd0c-4fbb-8be8-3291942543d1 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:58.398719 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:58.437077 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6431bcd-fd0c-4fbb-8be8-3291942543d1 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:58.471570 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645f5df7-7f41-46de-85e6-1108b694849d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:58.472207 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645f5df7-7f41-46de-85e6-1108b694849d None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:58.472488 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645f5df7-7f41-46de-85e6-1108b694849d None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:58.472757 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645f5df7-7f41-46de-85e6-1108b694849d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:58.482785 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645f5df7-7f41-46de-85e6-1108b694849d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:58.484243 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645f5df7-7f41-46de-85e6-1108b694849d None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:58.499033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-645f5df7-7f41-46de-85e6-1108b694849d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:03:58.499033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:03:58.499033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:03:58.499033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:03:58.499033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:03:58.499033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:03:58.499033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:03:58.499033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:03:58.499033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:03:58.499033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:03:58.499033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:03:58.499033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:03:58.499033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 01:03:58.499033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 01:03:58.499033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 01:03:58.499033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 01:03:58.500225 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:03:58.500225 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:03:58.500225 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:03:58.500225 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:03:58.500225 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:03:58.500225 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:03:58.500225 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:03:58.500225 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:03:58.500225 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:03:58.500225 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:03:58.500225 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:03:58.500225 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:03:58.500225 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:03:58.500225 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:03:58.500225 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:03:58.500225 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:03:58.500225 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 01:03:58.501144 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 01:03:58.501144 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 01:03:58.501144 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:03:58.501144 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 630/1283] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:03:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:03:58.505929 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34239673-3b3e-40d0-8ea2-ab9bbf1a28ac None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:58.601546 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34239673-3b3e-40d0-8ea2-ab9bbf1a28ac None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:58.634388 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8e94a3-dff6-492f-8c31-102660f4590d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:58.634982 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e94a3-dff6-492f-8c31-102660f4590d None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:58.635256 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e94a3-dff6-492f-8c31-102660f4590d None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:58.635525 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e94a3-dff6-492f-8c31-102660f4590d None admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:58.636178 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8e94a3-dff6-492f-8c31-102660f4590d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:03:58.636484 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8e94a3-dff6-492f-8c31-102660f4590d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:58.638115 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8e94a3-dff6-492f-8c31-102660f4590d None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:58.649747 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 654/1284] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Sep 8 01:03:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:03:58.655259 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645f5df7-7f41-46de-85e6-1108b694849d None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:58.752831 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645f5df7-7f41-46de-85e6-1108b694849d None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:58.767819 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:58.789385 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daa1bf6-ff05-45bd-905f-6fdf2e33afdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:58.789995 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa1bf6-ff05-45bd-905f-6fdf2e33afdd None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:58.790270 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa1bf6-ff05-45bd-905f-6fdf2e33afdd None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:58.790535 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa1bf6-ff05-45bd-905f-6fdf2e33afdd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:58.796817 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daa1bf6-ff05-45bd-905f-6fdf2e33afdd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:58.798241 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daa1bf6-ff05-45bd-905f-6fdf2e33afdd None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:58.809461 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application [None req-5daa1bf6-ff05-45bd-905f-6fdf2e33afdd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:03:58.809461 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:03:58.809461 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:03:58.809461 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:03:58.809461 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:03:58.809461 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:03:58.809461 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:03:58.809461 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:03:58.809461 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:03:58.809461 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:03:58.809461 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:03:58.809461 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:03:58.809461 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 01:03:58.809461 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 01:03:58.809461 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 01:03:58.810505 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 01:03:58.810505 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:03:58.810505 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:03:58.810505 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:03:58.810505 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:03:58.810505 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:03:58.810505 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:03:58.810505 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:03:58.810505 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application with Lock( Sep 08 01:03:58.810505 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:03:58.810505 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:03:58.810505 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:03:58.810505 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:03:58.810505 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:03:58.810505 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:03:58.810505 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:03:58.810505 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:03:58.811601 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 01:03:58.811601 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 01:03:58.811601 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 01:03:58.811601 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 01:03:58.811601 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 01:03:58.811601 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 01:03:58.811601 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 01:03:58.811601 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: ERROR keystone.server.flask.application Sep 08 01:03:58.811601 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 631/1285] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Sep 8 01:03:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:03:58.816316 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8e94a3-dff6-492f-8c31-102660f4590d None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:58.865184 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:03:58.913672 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8e94a3-dff6-492f-8c31-102660f4590d None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:58.945754 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2025bcf-dd7c-49b1-83cf-08e50900962a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:58.946348 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2025bcf-dd7c-49b1-83cf-08e50900962a None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:58.946626 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2025bcf-dd7c-49b1-83cf-08e50900962a None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:58.946897 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2025bcf-dd7c-49b1-83cf-08e50900962a None admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:58.947587 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2025bcf-dd7c-49b1-83cf-08e50900962a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:03:58.947894 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2025bcf-dd7c-49b1-83cf-08e50900962a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:58.949379 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2025bcf-dd7c-49b1-83cf-08e50900962a None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:58.961566 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 655/1286] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Sep 8 01:03:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:03:58.967634 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daa1bf6-ff05-45bd-905f-6fdf2e33afdd None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:03:59.063970 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daa1bf6-ff05-45bd-905f-6fdf2e33afdd None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:03:59.097953 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dafc88b-6c4d-4375-af36-013ffe435f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:03:59.098541 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dafc88b-6c4d-4375-af36-013ffe435f7c None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:03:59.098903 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dafc88b-6c4d-4375-af36-013ffe435f7c None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:03:59.099184 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dafc88b-6c4d-4375-af36-013ffe435f7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:03:59.099886 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dafc88b-6c4d-4375-af36-013ffe435f7c None admin] RBAC: Adding query filter params (role.id=34de369ec7c54715aa83a9052a26c3f2, scope.project.id=78ad2a8fb5264fb4af935e039d28c8de, user.id=301d30bbd3de4cc69e96bb12729442c6) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:03:59.100218 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dafc88b-6c4d-4375-af36-013ffe435f7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:03:59.101642 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dafc88b-6c4d-4375-af36-013ffe435f7c None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:03:59.107101 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 632/1287] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Sep 8 01:03:58 2022] GET /identity/v3/role_assignments?user.id=301d30bbd3de4cc69e96bb12729442c6&scope.project.id=78ad2a8fb5264fb4af935e039d28c8de&role.id=34de369ec7c54715aa83a9052a26c3f2 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:04:01.247805 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99c7e54-b549-4ebc-b26a-bd724dc91857 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:04:01.248394 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99c7e54-b549-4ebc-b26a-bd724dc91857 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:04:01.248394 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99c7e54-b549-4ebc-b26a-bd724dc91857 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:04:01.249700 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 656/1288] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:04:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:04:01.254788 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79cfce6-b855-47d1-a48a-2a490ff79de2 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:04:01.255779 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79cfce6-b855-47d1-a48a-2a490ff79de2 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:04:01.256188 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79cfce6-b855-47d1-a48a-2a490ff79de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:04:01.274323 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-c79cfce6-b855-47d1-a48a-2a490ff79de2 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:04:01.275494 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-c79cfce6-b855-47d1-a48a-2a490ff79de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:04:01.304603 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:04:01.340478 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 633/1289] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:04:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 01:04:01.606769 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99c7e54-b549-4ebc-b26a-bd724dc91857 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:04:01.706612 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99c7e54-b549-4ebc-b26a-bd724dc91857 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:04:01.717216 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:04:01.740616 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d9fb2-fd60-451d-8fc1-9f04e79fc168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:04:01.741227 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d9fb2-fd60-451d-8fc1-9f04e79fc168 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:04:01.741501 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d9fb2-fd60-451d-8fc1-9f04e79fc168 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:04:01.741766 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d9fb2-fd60-451d-8fc1-9f04e79fc168 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:04:01.742334 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954d9fb2-fd60-451d-8fc1-9f04e79fc168 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:04:01.743679 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954d9fb2-fd60-451d-8fc1-9f04e79fc168 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:04:01.752553 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-954d9fb2-fd60-451d-8fc1-9f04e79fc168 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 01:04:01.752553 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:04:01.752553 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 01:04:01.752553 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 01:04:01.752553 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 01:04:01.752553 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 01:04:01.752553 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 01:04:01.752553 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 01:04:01.752553 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 01:04:01.752553 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:04:01.752553 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 01:04:01.752553 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:04:01.752553 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:04:01.752553 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:04:01.752553 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:04:01.752553 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:04:01.752553 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:04:01.753790 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:04:01.753790 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:04:01.753790 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:04:01.753790 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:04:01.753790 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:04:01.753790 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:04:01.753790 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 01:04:01.753790 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 01:04:01.753790 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 01:04:01.753790 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 01:04:01.753790 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:04:01.753790 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:04:01.753790 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:04:01.753790 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:04:01.753790 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:04:01.753790 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:04:01.754682 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:04:01.754682 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:04:01.754682 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:04:01.754682 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:04:01.754682 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:04:01.754682 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:04:01.754682 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:04:01.754682 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:04:01.754682 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:04:01.754682 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:04:01.754682 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 01:04:01.754682 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 01:04:01.754682 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 01:04:01.754682 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:04:01.754682 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 657/1290] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Sep 8 01:04:01 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:04:01.760539 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79cfce6-b855-47d1-a48a-2a490ff79de2 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:04:01.854986 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79cfce6-b855-47d1-a48a-2a490ff79de2 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:04:01.889969 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee4bf6-a704-4d05-b3a8-8753fda872d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:04:01.890499 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee4bf6-a704-4d05-b3a8-8753fda872d4 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:04:01.890776 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee4bf6-a704-4d05-b3a8-8753fda872d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:04:01.891048 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee4bf6-a704-4d05-b3a8-8753fda872d4 None admin] PATH_INFO: `/v3/services` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:04:01.891667 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eee4bf6-a704-4d05-b3a8-8753fda872d4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:04:01.891966 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eee4bf6-a704-4d05-b3a8-8753fda872d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:04:01.893361 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eee4bf6-a704-4d05-b3a8-8753fda872d4 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:04:01.898250 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 634/1291] 10.222.0.42 () {56 vars in 1116 bytes} [Thu Sep 8 01:04:01 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:04:01.904107 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d9fb2-fd60-451d-8fc1-9f04e79fc168 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:04:02.001554 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d9fb2-fd60-451d-8fc1-9f04e79fc168 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:04:02.034329 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb6497e-9ab9-4a84-b1e9-1db0a08ad0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:04:02.034922 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb6497e-9ab9-4a84-b1e9-1db0a08ad0c9 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:04:02.035198 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb6497e-9ab9-4a84-b1e9-1db0a08ad0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:04:02.035468 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb6497e-9ab9-4a84-b1e9-1db0a08ad0c9 None admin] PATH_INFO: `/v3/services` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:04:02.036687 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb6497e-9ab9-4a84-b1e9-1db0a08ad0c9 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:04:02.036993 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb6497e-9ab9-4a84-b1e9-1db0a08ad0c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:04:02.038340 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb6497e-9ab9-4a84-b1e9-1db0a08ad0c9 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:04:02.042597 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 658/1292] 10.222.0.42 () {56 vars in 1116 bytes} [Thu Sep 8 01:04:01 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:04:04.190600 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1de5cd-3f29-43c0-9a65-68193838d419 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:04:04.191150 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1de5cd-3f29-43c0-9a65-68193838d419 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:04:04.191150 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1de5cd-3f29-43c0-9a65-68193838d419 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:04:04.193174 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 635/1293] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:04:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:04:04.198569 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf9d0b9-ba53-438d-be48-45c7bdd3931b None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:04:04.198958 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf9d0b9-ba53-438d-be48-45c7bdd3931b None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:04:04.199109 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf9d0b9-ba53-438d-be48-45c7bdd3931b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:04:04.217078 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-2bf9d0b9-ba53-438d-be48-45c7bdd3931b None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:04:04.218281 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-2bf9d0b9-ba53-438d-be48-45c7bdd3931b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:04:04.247511 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:04:04.285231 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 659/1294] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:04:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 01:04:04.553851 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1de5cd-3f29-43c0-9a65-68193838d419 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:04:04.584986 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:04:04.653521 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1de5cd-3f29-43c0-9a65-68193838d419 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:04:04.685869 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922aae7b-9214-4420-b542-427c81421a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:04:04.686460 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922aae7b-9214-4420-b542-427c81421a0b None admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:04:04.686733 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922aae7b-9214-4420-b542-427c81421a0b None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:04:04.687001 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922aae7b-9214-4420-b542-427c81421a0b None admin] PATH_INFO: `/v3/services` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:04:04.687628 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922aae7b-9214-4420-b542-427c81421a0b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:04:04.689013 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922aae7b-9214-4420-b542-427c81421a0b None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:04:04.703253 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 636/1295] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Sep 8 01:04:04 2022] POST /identity/v3/services => generated 246 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:04:06.892519 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7f391d-8dfb-408f-b1fc-6eb6b03a911c None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:04:06.893016 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7f391d-8dfb-408f-b1fc-6eb6b03a911c None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:04:06.893016 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7f391d-8dfb-408f-b1fc-6eb6b03a911c None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:04:06.894313 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 660/1296] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:04:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:04:06.899731 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570bec1-3e51-47d3-b9f2-e51a9fd45821 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:04:06.900169 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570bec1-3e51-47d3-b9f2-e51a9fd45821 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:04:06.900318 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570bec1-3e51-47d3-b9f2-e51a9fd45821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:04:06.918246 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-6570bec1-3e51-47d3-b9f2-e51a9fd45821 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:04:06.919429 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-6570bec1-3e51-47d3-b9f2-e51a9fd45821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:04:06.991399 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 637/1297] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:04:06 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 01:04:07.265023 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7f391d-8dfb-408f-b1fc-6eb6b03a911c None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:04:07.361894 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7f391d-8dfb-408f-b1fc-6eb6b03a911c None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:04:07.394615 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e6c863-d93e-4bc5-b753-0f5f73c00448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:04:07.395277 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6c863-d93e-4bc5-b753-0f5f73c00448 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:04:07.395546 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6c863-d93e-4bc5-b753-0f5f73c00448 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:04:07.395819 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6c863-d93e-4bc5-b753-0f5f73c00448 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:04:07.396420 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e6c863-d93e-4bc5-b753-0f5f73c00448 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:04:07.398050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e6c863-d93e-4bc5-b753-0f5f73c00448 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:04:07.406197 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-c1e6c863-d93e-4bc5-b753-0f5f73c00448 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 01:04:07.406197 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:04:07.406197 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 01:04:07.406197 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 01:04:07.406197 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 01:04:07.406197 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 01:04:07.406197 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 01:04:07.406197 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 01:04:07.406197 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 01:04:07.406197 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:04:07.406197 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 01:04:07.406197 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:04:07.406197 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:04:07.406197 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:04:07.406197 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:04:07.406197 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:04:07.406197 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:04:07.407389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:04:07.407389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:04:07.407389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:04:07.407389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:04:07.407389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:04:07.407389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:04:07.407389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 01:04:07.407389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 01:04:07.407389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 01:04:07.407389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 01:04:07.407389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:04:07.407389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:04:07.407389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:04:07.407389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:04:07.407389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:04:07.407389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:04:07.408589 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:04:07.408589 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:04:07.408589 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:04:07.408589 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:04:07.408589 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:04:07.408589 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:04:07.408589 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:04:07.408589 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:04:07.408589 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:04:07.408589 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:04:07.408589 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 01:04:07.408589 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 01:04:07.408589 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 01:04:07.408589 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:04:07.408589 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 661/1298] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Sep 8 01:04:07 2022] GET /identity/v3/services/placement => generated 90 bytes in 143 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:04:07.414391 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6570bec1-3e51-47d3-b9f2-e51a9fd45821 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:04:07.433409 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:04:07.513530 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6570bec1-3e51-47d3-b9f2-e51a9fd45821 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:04:07.546372 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e51063-4c97-49ef-ae70-45b10a975092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:04:07.546906 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e51063-4c97-49ef-ae70-45b10a975092 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:04:07.547185 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e51063-4c97-49ef-ae70-45b10a975092 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:04:07.547456 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e51063-4c97-49ef-ae70-45b10a975092 None admin] PATH_INFO: `/v3/services` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:04:07.548075 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e51063-4c97-49ef-ae70-45b10a975092 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:04:07.548403 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e51063-4c97-49ef-ae70-45b10a975092 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:04:07.549768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e51063-4c97-49ef-ae70-45b10a975092 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:04:07.555352 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 638/1299] 10.222.0.42 () {56 vars in 1116 bytes} [Thu Sep 8 01:04:07 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:04:07.561222 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e6c863-d93e-4bc5-b753-0f5f73c00448 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:04:07.592310 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:04:07.660250 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e6c863-d93e-4bc5-b753-0f5f73c00448 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:04:07.693468 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c512f32-431e-46e4-96ad-3d0aae46283b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:04:07.694063 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c512f32-431e-46e4-96ad-3d0aae46283b None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:04:07.694340 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c512f32-431e-46e4-96ad-3d0aae46283b None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:04:07.694611 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c512f32-431e-46e4-96ad-3d0aae46283b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:04:07.695307 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c512f32-431e-46e4-96ad-3d0aae46283b None admin] RBAC: Adding query filter params (interface=public, service_id=5c3827a99fdf4422a56cae1fe4dc88ec, region_id=RegionOne) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:04:07.695616 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c512f32-431e-46e4-96ad-3d0aae46283b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:04:07.697005 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c512f32-431e-46e4-96ad-3d0aae46283b None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:04:07.701853 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 662/1300] 10.222.0.42 () {56 vars in 1251 bytes} [Thu Sep 8 01:04:07 2022] GET /identity/v3/endpoints?service_id=5c3827a99fdf4422a56cae1fe4dc88ec&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:04:07.708118 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e51063-4c97-49ef-ae70-45b10a975092 None admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:04:07.788310 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:04:07.806066 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e51063-4c97-49ef-ae70-45b10a975092 None admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:04:07.838623 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0385d243-cab2-43b4-a6f3-fc229c2cabf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:04:07.839215 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0385d243-cab2-43b4-a6f3-fc229c2cabf4 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:04:07.839493 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0385d243-cab2-43b4-a6f3-fc229c2cabf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:04:07.839763 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0385d243-cab2-43b4-a6f3-fc229c2cabf4 None admin] PATH_INFO: `/v3/services` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:04:07.840379 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0385d243-cab2-43b4-a6f3-fc229c2cabf4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:04:07.841749 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0385d243-cab2-43b4-a6f3-fc229c2cabf4 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:04:07.848359 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 639/1301] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Sep 8 01:04:07 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:04:09.976922 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1236f7-655e-4bd6-8888-e0c9386ff163 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:04:09.977443 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1236f7-655e-4bd6-8888-e0c9386ff163 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:04:09.977443 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1236f7-655e-4bd6-8888-e0c9386ff163 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:04:09.978723 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 663/1302] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:04:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:04:09.983915 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de1004-c405-4095-b101-452ab0b7cd0a None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:04:09.984352 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de1004-c405-4095-b101-452ab0b7cd0a None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:04:09.984493 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de1004-c405-4095-b101-452ab0b7cd0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:04:10.002735 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-c8de1004-c405-4095-b101-452ab0b7cd0a None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:04:10.003915 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-c8de1004-c405-4095-b101-452ab0b7cd0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:04:10.069016 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 640/1303] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:04:09 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 01:04:10.336097 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1236f7-655e-4bd6-8888-e0c9386ff163 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:04:10.430587 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:04:10.437437 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1236f7-655e-4bd6-8888-e0c9386ff163 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:04:10.469144 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1026b9a7-319c-4df2-9f6a-755163a4fab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:04:10.469758 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1026b9a7-319c-4df2-9f6a-755163a4fab7 None admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:04:10.470033 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1026b9a7-319c-4df2-9f6a-755163a4fab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:04:10.470302 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1026b9a7-319c-4df2-9f6a-755163a4fab7 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:04:10.470877 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1026b9a7-319c-4df2-9f6a-755163a4fab7 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:04:10.472257 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1026b9a7-319c-4df2-9f6a-755163a4fab7 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:04:10.481358 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application [None req-1026b9a7-319c-4df2-9f6a-755163a4fab7 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 01:04:10.481358 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:04:10.481358 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 01:04:10.481358 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 01:04:10.481358 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 01:04:10.481358 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 01:04:10.481358 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 01:04:10.481358 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 01:04:10.481358 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 01:04:10.481358 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:04:10.481358 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 01:04:10.481358 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:04:10.481358 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 01:04:10.481358 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 01:04:10.481358 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 01:04:10.481358 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 01:04:10.481358 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 01:04:10.482592 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 01:04:10.482592 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 01:04:10.482592 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 01:04:10.482592 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 01:04:10.482592 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 01:04:10.482592 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 01:04:10.482592 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 01:04:10.482592 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 01:04:10.482592 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 01:04:10.482592 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 01:04:10.482592 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 01:04:10.482592 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 01:04:10.482592 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 01:04:10.482592 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 01:04:10.482592 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 01:04:10.482592 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 01:04:10.483520 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 01:04:10.483520 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application with Lock( Sep 08 01:04:10.483520 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 01:04:10.483520 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self._enter() Sep 08 01:04:10.483520 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 01:04:10.483520 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 01:04:10.483520 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 01:04:10.483520 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application return self.creator() Sep 08 01:04:10.483520 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 01:04:10.483520 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application created_value = creator( Sep 08 01:04:10.483520 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 01:04:10.483520 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 01:04:10.483520 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 01:04:10.483520 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: ERROR keystone.server.flask.application Sep 08 01:04:10.487252 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 664/1304] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Sep 8 01:04:10 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 01:04:10.488638 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8de1004-c405-4095-b101-452ab0b7cd0a None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:04:10.562495 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:04:10.585574 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8de1004-c405-4095-b101-452ab0b7cd0a None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:04:10.618562 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ebca93-5f56-4e2c-a1fe-3a54cc5c0cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:04:10.619096 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ebca93-5f56-4e2c-a1fe-3a54cc5c0cd7 None admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:04:10.619368 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ebca93-5f56-4e2c-a1fe-3a54cc5c0cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:04:10.619636 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ebca93-5f56-4e2c-a1fe-3a54cc5c0cd7 None admin] PATH_INFO: `/v3/services` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:04:10.620288 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ebca93-5f56-4e2c-a1fe-3a54cc5c0cd7 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:04:10.620586 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ebca93-5f56-4e2c-a1fe-3a54cc5c0cd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:04:10.621952 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ebca93-5f56-4e2c-a1fe-3a54cc5c0cd7 None admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:04:10.627024 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 641/1305] 10.222.0.42 () {56 vars in 1116 bytes} [Thu Sep 8 01:04:10 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:04:10.633607 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1026b9a7-319c-4df2-9f6a-755163a4fab7 None admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:04:10.730522 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1026b9a7-319c-4df2-9f6a-755163a4fab7 None admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:04:10.764312 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b905a-947b-434a-a7b6-fe493c8a0eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:04:10.764911 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b905a-947b-434a-a7b6-fe493c8a0eb3 None admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:04:10.765195 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b905a-947b-434a-a7b6-fe493c8a0eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:04:10.765470 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b905a-947b-434a-a7b6-fe493c8a0eb3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:04:10.766138 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3b905a-947b-434a-a7b6-fe493c8a0eb3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:04:10.767504 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3b905a-947b-434a-a7b6-fe493c8a0eb3 None admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:04:10.786572 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 665/1306] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:04:10 2022] POST /identity/v3/endpoints => generated 331 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:04:20.624110 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=30 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:04:20.782506 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:05:15.095035 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e962c-2155-44c7-852d-fdb201e72660 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:05:15.095610 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e962c-2155-44c7-852d-fdb201e72660 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:05:15.095610 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e962c-2155-44c7-852d-fdb201e72660 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:05:15.103418 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e2425-fe6f-4aad-a1d7-cc4509e1093e None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:05:15.103815 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e2425-fe6f-4aad-a1d7-cc4509e1093e None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:05:15.104087 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e2425-fe6f-4aad-a1d7-cc4509e1093e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:05:15.106850 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 642/1307] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:05:15 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:05:15.302136 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-7f8e2425-fe6f-4aad-a1d7-cc4509e1093e None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:05:15.304863 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-7f8e2425-fe6f-4aad-a1d7-cc4509e1093e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:05:15.515874 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 666/1308] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:05:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:05:15.821816 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c97dd-246f-4ea1-9ea2-7a99756e91a0 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:05:15.822186 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c97dd-246f-4ea1-9ea2-7a99756e91a0 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:05:15.822361 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c97dd-246f-4ea1-9ea2-7a99756e91a0 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:05:15.823986 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 643/1309] 10.222.0.42 () {54 vars in 965 bytes} [Thu Sep 8 01:05:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:05:15.830990 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a48ebf-7cd4-41c9-ac83-51b70b2f26f8 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:05:15.831397 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a48ebf-7cd4-41c9-ac83-51b70b2f26f8 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:05:15.831560 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a48ebf-7cd4-41c9-ac83-51b70b2f26f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:05:15.945348 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-d4a48ebf-7cd4-41c9-ac83-51b70b2f26f8 None None] MFA Rules not processed for user `5dbbcc3dbcc745458e97b31c6781ce70`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:05:15.946621 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-d4a48ebf-7cd4-41c9-ac83-51b70b2f26f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:05:15.974544 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:05:16.043552 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 667/1310] 10.222.0.42 () {58 vars in 1062 bytes} [Thu Sep 8 01:05:15 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:05:16.053652 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c97dd-246f-4ea1-9ea2-7a99756e91a0 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:05:16.055782 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-7c1c97dd-246f-4ea1-9ea2-7a99756e91a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:05:16.188410 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c97dd-246f-4ea1-9ea2-7a99756e91a0 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:05:16.271351 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d56f073-70aa-4477-802b-b5a6f7d79855 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:05:16.271965 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56f073-70aa-4477-802b-b5a6f7d79855 service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:05:16.272379 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56f073-70aa-4477-802b-b5a6f7d79855 service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:05:16.272649 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56f073-70aa-4477-802b-b5a6f7d79855 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:05:16.274249 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-9d56f073-70aa-4477-802b-b5a6f7d79855 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:05:16.336747 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:05:16.405898 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d56f073-70aa-4477-802b-b5a6f7d79855 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:05:16.407406 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d56f073-70aa-4477-802b-b5a6f7d79855 service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:05:16.429933 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 644/1311] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:05:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:05:19.055264 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d23f4-1425-4b90-8c14-6ee1e0f8b043 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:05:19.055793 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d23f4-1425-4b90-8c14-6ee1e0f8b043 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:05:19.055793 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d23f4-1425-4b90-8c14-6ee1e0f8b043 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:05:19.057070 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 668/1312] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:05:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:05:19.063742 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa548b4-ebc1-433d-984a-1dfb4652d20c None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:05:19.064210 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa548b4-ebc1-433d-984a-1dfb4652d20c None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:05:19.064316 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa548b4-ebc1-433d-984a-1dfb4652d20c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:05:19.142349 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-efa548b4-ebc1-433d-984a-1dfb4652d20c None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:05:19.143604 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-efa548b4-ebc1-433d-984a-1dfb4652d20c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:05:19.248344 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 645/1313] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:05:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:05:19.538626 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658d23f4-1425-4b90-8c14-6ee1e0f8b043 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:05:19.540240 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-658d23f4-1425-4b90-8c14-6ee1e0f8b043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:05:19.644639 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658d23f4-1425-4b90-8c14-6ee1e0f8b043 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:05:19.717053 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:05:19.719410 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080e506e-e496-4de7-bbde-0be702a6d6c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:05:19.720217 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e506e-e496-4de7-bbde-0be702a6d6c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:05:19.720696 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e506e-e496-4de7-bbde-0be702a6d6c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:05:19.721519 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e506e-e496-4de7-bbde-0be702a6d6c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:05:19.723598 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-080e506e-e496-4de7-bbde-0be702a6d6c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:05:19.829668 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080e506e-e496-4de7-bbde-0be702a6d6c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:05:19.831553 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080e506e-e496-4de7-bbde-0be702a6d6c9 service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:05:19.850086 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 669/1314] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:05:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:05:22.293175 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee6229b-195c-41ee-b19e-1a34a2e0c66b None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:05:22.293734 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee6229b-195c-41ee-b19e-1a34a2e0c66b None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:05:22.293734 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee6229b-195c-41ee-b19e-1a34a2e0c66b None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:05:22.295045 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 646/1315] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:05:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:05:22.300680 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8856f244-b345-4c89-ad0c-38d7436d97f0 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:05:22.301090 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8856f244-b345-4c89-ad0c-38d7436d97f0 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:05:22.301226 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8856f244-b345-4c89-ad0c-38d7436d97f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:05:22.367479 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-8856f244-b345-4c89-ad0c-38d7436d97f0 None None] MFA Rules not processed for user `3c115932304948d4968c4f07fae9c96b`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:05:22.368616 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-8856f244-b345-4c89-ad0c-38d7436d97f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:05:22.420849 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:05:22.464953 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 670/1316] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:05:22 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:05:22.746717 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee6229b-195c-41ee-b19e-1a34a2e0c66b None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:05:22.748280 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-bee6229b-195c-41ee-b19e-1a34a2e0c66b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:05:22.757991 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:05:22.850524 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee6229b-195c-41ee-b19e-1a34a2e0c66b None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:05:22.922240 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af3d2c-4194-469b-a09a-8ae1722fcad2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:05:22.922763 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af3d2c-4194-469b-a09a-8ae1722fcad2 service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:05:22.923035 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af3d2c-4194-469b-a09a-8ae1722fcad2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:05:22.923301 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af3d2c-4194-469b-a09a-8ae1722fcad2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:05:22.925394 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-53af3d2c-4194-469b-a09a-8ae1722fcad2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:05:22.985784 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:05:23.032829 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53af3d2c-4194-469b-a09a-8ae1722fcad2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:05:23.034394 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53af3d2c-4194-469b-a09a-8ae1722fcad2 service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:05:23.052105 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 647/1317] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:05:22 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:05:25.747277 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174e1078-5674-4fe7-bfc4-5448e01af2bb None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:05:25.747806 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174e1078-5674-4fe7-bfc4-5448e01af2bb None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:05:25.747806 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174e1078-5674-4fe7-bfc4-5448e01af2bb None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:05:25.749185 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 671/1318] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:05:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:05:25.754650 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9ee4c-21a4-4e61-a32d-057a2aaecac3 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:05:25.755060 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9ee4c-21a4-4e61-a32d-057a2aaecac3 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:05:25.755195 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9ee4c-21a4-4e61-a32d-057a2aaecac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:05:25.821073 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-76f9ee4c-21a4-4e61-a32d-057a2aaecac3 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:05:25.822263 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-76f9ee4c-21a4-4e61-a32d-057a2aaecac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:05:25.918914 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 648/1319] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:05:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:05:26.203290 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174e1078-5674-4fe7-bfc4-5448e01af2bb None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:05:26.205467 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-174e1078-5674-4fe7-bfc4-5448e01af2bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:05:26.308499 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174e1078-5674-4fe7-bfc4-5448e01af2bb None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:05:26.382882 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3883404e-d5ec-430f-a06f-c03f29739cb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:05:26.383483 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883404e-d5ec-430f-a06f-c03f29739cb2 service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:05:26.383760 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883404e-d5ec-430f-a06f-c03f29739cb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:05:26.384029 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883404e-d5ec-430f-a06f-c03f29739cb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:05:26.385666 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-3883404e-d5ec-430f-a06f-c03f29739cb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:05:26.392809 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:05:26.496458 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3883404e-d5ec-430f-a06f-c03f29739cb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:05:26.497896 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3883404e-d5ec-430f-a06f-c03f29739cb2 service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:05:26.516536 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 672/1320] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:05:26 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:05:29.314912 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a061d-8d5b-47e1-9cf5-8cda3bcd32bf None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:05:29.315482 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a061d-8d5b-47e1-9cf5-8cda3bcd32bf None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:05:29.315482 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a061d-8d5b-47e1-9cf5-8cda3bcd32bf None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:05:29.316829 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 649/1321] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:05:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:05:29.322066 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c65e20-3bb3-4104-9bf4-cb52b2f15481 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:05:29.322486 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c65e20-3bb3-4104-9bf4-cb52b2f15481 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:05:29.322671 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c65e20-3bb3-4104-9bf4-cb52b2f15481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:05:29.395529 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-68c65e20-3bb3-4104-9bf4-cb52b2f15481 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:05:29.396829 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-68c65e20-3bb3-4104-9bf4-cb52b2f15481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:05:29.458415 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:05:29.501601 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 673/1322] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:05:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:05:29.786865 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485a061d-8d5b-47e1-9cf5-8cda3bcd32bf None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:05:29.788437 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-485a061d-8d5b-47e1-9cf5-8cda3bcd32bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:05:29.809388 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:05:29.894088 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485a061d-8d5b-47e1-9cf5-8cda3bcd32bf None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:05:29.967236 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd1dca2-9ad3-4100-a379-10072a242adf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:05:29.967846 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1dca2-9ad3-4100-a379-10072a242adf service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:05:29.968126 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1dca2-9ad3-4100-a379-10072a242adf service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:05:29.968424 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1dca2-9ad3-4100-a379-10072a242adf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:05:29.970023 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-ebd1dca2-9ad3-4100-a379-10072a242adf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:05:30.041874 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:05:30.080079 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd1dca2-9ad3-4100-a379-10072a242adf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:05:30.081531 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd1dca2-9ad3-4100-a379-10072a242adf service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:05:30.100271 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 650/1323] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:05:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:05:32.320725 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54128cb-6bcc-4d13-b7d0-b97f8364bfd3 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:05:32.321290 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54128cb-6bcc-4d13-b7d0-b97f8364bfd3 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:05:32.321388 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54128cb-6bcc-4d13-b7d0-b97f8364bfd3 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:05:32.322855 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 674/1324] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:05:32.328199 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b592bf-c12b-4202-8564-33aa7b28039e None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:05:32.328640 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b592bf-c12b-4202-8564-33aa7b28039e None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:05:32.328884 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b592bf-c12b-4202-8564-33aa7b28039e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:05:32.395280 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-31b592bf-c12b-4202-8564-33aa7b28039e None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:05:32.396436 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-31b592bf-c12b-4202-8564-33aa7b28039e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:05:32.500956 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 651/1325] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:05:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:05:32.782287 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54128cb-6bcc-4d13-b7d0-b97f8364bfd3 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:05:32.784067 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-a54128cb-6bcc-4d13-b7d0-b97f8364bfd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:05:32.886268 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54128cb-6bcc-4d13-b7d0-b97f8364bfd3 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:05:32.960498 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab035f2-5d5b-450f-a605-ab68441c5e50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:05:32.961101 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab035f2-5d5b-450f-a605-ab68441c5e50 service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:05:32.961379 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab035f2-5d5b-450f-a605-ab68441c5e50 service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:05:32.961703 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab035f2-5d5b-450f-a605-ab68441c5e50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:05:32.963306 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-7ab035f2-5d5b-450f-a605-ab68441c5e50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:05:32.980968 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:05:33.075251 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab035f2-5d5b-450f-a605-ab68441c5e50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:05:33.076653 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab035f2-5d5b-450f-a605-ab68441c5e50 service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:05:33.095164 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 675/1326] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:05:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:05:35.309838 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe17e34a-b0db-4dee-beb9-d806a7c3046b None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:05:35.310387 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe17e34a-b0db-4dee-beb9-d806a7c3046b None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:05:35.310387 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe17e34a-b0db-4dee-beb9-d806a7c3046b None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:05:35.311729 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 652/1327] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:05:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:05:35.318205 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014da28-7db2-42e5-958a-5653239c4ba3 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:05:35.318656 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014da28-7db2-42e5-958a-5653239c4ba3 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:05:35.318753 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014da28-7db2-42e5-958a-5653239c4ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:05:35.385719 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-2014da28-7db2-42e5-958a-5653239c4ba3 None None] MFA Rules not processed for user `3c115932304948d4968c4f07fae9c96b`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:05:35.387126 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-2014da28-7db2-42e5-958a-5653239c4ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:05:35.453080 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:05:35.484398 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 676/1328] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:05:35 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:05:35.785115 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde53a70-9417-47a6-ab75-e2ce2f5369ad None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:05:35.785499 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde53a70-9417-47a6-ab75-e2ce2f5369ad None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:05:35.785847 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde53a70-9417-47a6-ab75-e2ce2f5369ad None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:05:35.787272 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 653/1329] 10.222.0.42 () {54 vars in 965 bytes} [Thu Sep 8 01:05:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:05:35.793604 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39acdf-68c2-43db-bf53-6d74545c8db2 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:05:35.794094 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39acdf-68c2-43db-bf53-6d74545c8db2 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:05:35.794369 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39acdf-68c2-43db-bf53-6d74545c8db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:05:35.876124 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-1c39acdf-68c2-43db-bf53-6d74545c8db2 None None] MFA Rules not processed for user `5dbbcc3dbcc745458e97b31c6781ce70`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:05:35.877404 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-1c39acdf-68c2-43db-bf53-6d74545c8db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:05:35.952627 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 677/1330] 10.222.0.42 () {58 vars in 1062 bytes} [Thu Sep 8 01:05:35 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:05:35.961717 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde53a70-9417-47a6-ab75-e2ce2f5369ad None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:05:35.963384 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-fde53a70-9417-47a6-ab75-e2ce2f5369ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:05:35.995392 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:05:36.069668 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde53a70-9417-47a6-ab75-e2ce2f5369ad None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:05:36.145474 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655b43e0-061a-4e12-bf31-14a9409bf690 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:05:36.145990 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b43e0-061a-4e12-bf31-14a9409bf690 service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:05:36.146267 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b43e0-061a-4e12-bf31-14a9409bf690 service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:05:36.146538 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b43e0-061a-4e12-bf31-14a9409bf690 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:05:36.148124 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-655b43e0-061a-4e12-bf31-14a9409bf690 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:05:36.226122 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:05:36.258617 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655b43e0-061a-4e12-bf31-14a9409bf690 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:05:36.260039 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655b43e0-061a-4e12-bf31-14a9409bf690 service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:05:36.279186 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 654/1331] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:05:35 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:05:39.067582 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a907c55-698b-45bf-8db3-b6f3b639676a None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:05:39.068113 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a907c55-698b-45bf-8db3-b6f3b639676a None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:05:39.068113 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a907c55-698b-45bf-8db3-b6f3b639676a None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:05:39.069494 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 678/1332] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:05:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:05:39.074729 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c004971-2882-4c82-9d90-2d96eba86900 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:05:39.075138 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c004971-2882-4c82-9d90-2d96eba86900 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:05:39.075267 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c004971-2882-4c82-9d90-2d96eba86900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:05:39.139891 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-2c004971-2882-4c82-9d90-2d96eba86900 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:05:39.141122 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-2c004971-2882-4c82-9d90-2d96eba86900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:05:39.240062 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 655/1333] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:05:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:05:39.520196 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a907c55-698b-45bf-8db3-b6f3b639676a None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:05:39.521778 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-5a907c55-698b-45bf-8db3-b6f3b639676a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:05:39.592524 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:05:39.627604 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a907c55-698b-45bf-8db3-b6f3b639676a None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:05:39.700540 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fcd0b9-db1b-4124-ba76-27242701dbfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:05:39.701147 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fcd0b9-db1b-4124-ba76-27242701dbfb service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:05:39.701426 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fcd0b9-db1b-4124-ba76-27242701dbfb service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:05:39.701723 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fcd0b9-db1b-4124-ba76-27242701dbfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:05:39.703344 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-62fcd0b9-db1b-4124-ba76-27242701dbfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:05:39.812740 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fcd0b9-db1b-4124-ba76-27242701dbfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:05:39.814142 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fcd0b9-db1b-4124-ba76-27242701dbfb service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:05:39.823913 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:05:39.832485 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 679/1334] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:05:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:05:42.021737 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b903bc3-5923-4368-a6e5-18872edb363c None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:05:42.022246 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b903bc3-5923-4368-a6e5-18872edb363c None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:05:42.022361 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b903bc3-5923-4368-a6e5-18872edb363c None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:05:42.023654 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 656/1335] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:05:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:05:42.028957 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318637eb-4919-412a-a69b-15b5e9c4ba50 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:05:42.029382 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318637eb-4919-412a-a69b-15b5e9c4ba50 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:05:42.029493 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318637eb-4919-412a-a69b-15b5e9c4ba50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:05:42.096007 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-318637eb-4919-412a-a69b-15b5e9c4ba50 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:05:42.097253 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-318637eb-4919-412a-a69b-15b5e9c4ba50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:05:42.194001 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 680/1336] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:05:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:05:42.481543 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b903bc3-5923-4368-a6e5-18872edb363c None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:05:42.483123 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-6b903bc3-5923-4368-a6e5-18872edb363c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:05:42.524817 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:05:42.588050 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b903bc3-5923-4368-a6e5-18872edb363c None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:05:42.664089 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cd14d4-e457-4683-8249-2a56c26f18a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:05:42.664724 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd14d4-e457-4683-8249-2a56c26f18a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:05:42.665004 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd14d4-e457-4683-8249-2a56c26f18a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:05:42.665275 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd14d4-e457-4683-8249-2a56c26f18a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:05:42.666866 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-39cd14d4-e457-4683-8249-2a56c26f18a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:05:42.749604 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:05:42.776830 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cd14d4-e457-4683-8249-2a56c26f18a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:05:42.778314 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cd14d4-e457-4683-8249-2a56c26f18a9 service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:05:42.796685 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 657/1337] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:05:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:05:45.269398 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcbba52-294b-48cc-9d1b-22df36c25388 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:05:45.270017 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcbba52-294b-48cc-9d1b-22df36c25388 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:05:45.270017 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcbba52-294b-48cc-9d1b-22df36c25388 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:05:45.271269 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 681/1338] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:05:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:05:45.276562 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a48bd-bdc2-4ec2-9509-0e69b72efe9d None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:05:45.276959 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a48bd-bdc2-4ec2-9509-0e69b72efe9d None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:05:45.277147 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a48bd-bdc2-4ec2-9509-0e69b72efe9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:05:45.342713 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-715a48bd-bdc2-4ec2-9509-0e69b72efe9d None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:05:45.343953 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-715a48bd-bdc2-4ec2-9509-0e69b72efe9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:05:45.440685 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 658/1339] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:05:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:05:45.721465 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcbba52-294b-48cc-9d1b-22df36c25388 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:05:45.723280 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-bfcbba52-294b-48cc-9d1b-22df36c25388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:05:45.798667 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:05:45.825618 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcbba52-294b-48cc-9d1b-22df36c25388 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:05:45.899491 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b332f7db-eadb-49ce-8211-886e69dcbafd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:05:45.900024 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b332f7db-eadb-49ce-8211-886e69dcbafd service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:05:45.900342 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b332f7db-eadb-49ce-8211-886e69dcbafd service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:05:45.900617 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b332f7db-eadb-49ce-8211-886e69dcbafd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:05:45.902212 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-b332f7db-eadb-49ce-8211-886e69dcbafd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:05:46.009406 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b332f7db-eadb-49ce-8211-886e69dcbafd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:05:46.010838 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b332f7db-eadb-49ce-8211-886e69dcbafd service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:05:46.029441 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 682/1340] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:05:45 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:05:50.265434 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0904fc-646a-4094-a8ab-3a5d60c04018 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:05:50.265969 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0904fc-646a-4094-a8ab-3a5d60c04018 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:05:50.265969 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0904fc-646a-4094-a8ab-3a5d60c04018 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:05:50.267433 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 659/1341] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:05:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:05:50.272739 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e393685-c266-4afa-9a14-b732b97b3653 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:05:50.273150 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e393685-c266-4afa-9a14-b732b97b3653 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:05:50.273284 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e393685-c266-4afa-9a14-b732b97b3653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:05:50.278921 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:05:50.341843 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-2e393685-c266-4afa-9a14-b732b97b3653 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:05:50.342974 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-2e393685-c266-4afa-9a14-b732b97b3653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:05:50.441492 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 683/1342] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:05:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:05:50.726613 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0904fc-646a-4094-a8ab-3a5d60c04018 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:05:50.728278 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-8e0904fc-646a-4094-a8ab-3a5d60c04018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:05:50.777851 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:05:50.834909 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0904fc-646a-4094-a8ab-3a5d60c04018 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:05:50.910059 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb7fc19-5087-4a19-889f-12ebb063857c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:05:50.910587 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb7fc19-5087-4a19-889f-12ebb063857c service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:05:50.910867 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb7fc19-5087-4a19-889f-12ebb063857c service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:05:50.911140 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb7fc19-5087-4a19-889f-12ebb063857c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:05:50.912772 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-9cb7fc19-5087-4a19-889f-12ebb063857c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:05:51.004506 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:05:51.022713 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb7fc19-5087-4a19-889f-12ebb063857c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:05:51.024186 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb7fc19-5087-4a19-889f-12ebb063857c service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:05:51.042812 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 660/1343] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:05:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:05:53.833419 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211d5bd1-b11a-418a-bb6f-1923f041e1b9 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:05:53.833991 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211d5bd1-b11a-418a-bb6f-1923f041e1b9 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:05:53.833991 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211d5bd1-b11a-418a-bb6f-1923f041e1b9 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:05:53.835325 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 684/1344] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:05:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:05:53.840484 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe2798c-782d-40d9-aad7-45b6c54fc2db None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:05:53.840893 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe2798c-782d-40d9-aad7-45b6c54fc2db None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:05:53.841041 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe2798c-782d-40d9-aad7-45b6c54fc2db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:05:53.909267 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-4fe2798c-782d-40d9-aad7-45b6c54fc2db None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:05:53.910600 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-4fe2798c-782d-40d9-aad7-45b6c54fc2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:05:54.008896 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 661/1345] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:05:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:05:54.303150 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211d5bd1-b11a-418a-bb6f-1923f041e1b9 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:05:54.304859 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-211d5bd1-b11a-418a-bb6f-1923f041e1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:05:54.397271 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:05:54.415541 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211d5bd1-b11a-418a-bb6f-1923f041e1b9 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:05:54.495976 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379d7cbe-f2c2-4e89-b1ae-5572c87e687c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:05:54.496608 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d7cbe-f2c2-4e89-b1ae-5572c87e687c service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:05:54.496888 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d7cbe-f2c2-4e89-b1ae-5572c87e687c service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:05:54.497158 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d7cbe-f2c2-4e89-b1ae-5572c87e687c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:05:54.498753 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-379d7cbe-f2c2-4e89-b1ae-5572c87e687c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:05:54.615309 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379d7cbe-f2c2-4e89-b1ae-5572c87e687c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:05:54.616796 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379d7cbe-f2c2-4e89-b1ae-5572c87e687c service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:05:54.641191 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 685/1346] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:05:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:05:58.763768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f41660-652a-4439-8731-872ef3df159b None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:05:58.764295 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f41660-652a-4439-8731-872ef3df159b None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:05:58.764387 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f41660-652a-4439-8731-872ef3df159b None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:05:58.765698 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 662/1347] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:05:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:05:58.771847 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c5786-f9c7-42f2-8d63-88ec35792245 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:05:58.772317 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c5786-f9c7-42f2-8d63-88ec35792245 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:05:58.772407 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c5786-f9c7-42f2-8d63-88ec35792245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:05:58.788899 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:05:58.851638 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-a71c5786-f9c7-42f2-8d63-88ec35792245 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:05:58.853025 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-a71c5786-f9c7-42f2-8d63-88ec35792245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:05:58.960254 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 686/1348] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:05:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:05:59.249708 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f41660-652a-4439-8731-872ef3df159b None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:05:59.251289 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-04f41660-652a-4439-8731-872ef3df159b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:05:59.310121 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:05:59.355376 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f41660-652a-4439-8731-872ef3df159b None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:05:59.429998 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992f5549-1b2b-4b58-98dd-ac514713e6a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:05:59.430601 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f5549-1b2b-4b58-98dd-ac514713e6a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:05:59.430881 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f5549-1b2b-4b58-98dd-ac514713e6a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:05:59.431151 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f5549-1b2b-4b58-98dd-ac514713e6a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:05:59.432801 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-992f5549-1b2b-4b58-98dd-ac514713e6a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:05:59.537670 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:05:59.541621 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992f5549-1b2b-4b58-98dd-ac514713e6a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:05:59.543063 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992f5549-1b2b-4b58-98dd-ac514713e6a2 service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:05:59.561523 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 663/1349] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:05:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:06:04.440961 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25acd3cc-e33e-4cd1-bbfc-cd39ae7422dd None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:04.441424 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25acd3cc-e33e-4cd1-bbfc-cd39ae7422dd None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:04.441548 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25acd3cc-e33e-4cd1-bbfc-cd39ae7422dd None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:04.443047 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 687/1350] 10.222.0.42 () {54 vars in 921 bytes} [Thu Sep 8 01:06:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:06:04.447985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de22e51-01f6-4015-abfb-3e9ef102f085 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:04.448759 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de22e51-01f6-4015-abfb-3e9ef102f085 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:04.449183 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de22e51-01f6-4015-abfb-3e9ef102f085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:04.552693 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-5de22e51-01f6-4015-abfb-3e9ef102f085 None None] MFA Rules not processed for user `301d30bbd3de4cc69e96bb12729442c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:06:04.552693 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-5de22e51-01f6-4015-abfb-3e9ef102f085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:04.721803 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 664/1351] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Sep 8 01:06:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:06:08.903010 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ba0dc6-8dc3-453a-ab79-7716ad957c24 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:08.903855 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ba0dc6-8dc3-453a-ab79-7716ad957c24 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:08.904161 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ba0dc6-8dc3-453a-ab79-7716ad957c24 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:08.905515 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 688/1352] 10.222.0.42 () {54 vars in 921 bytes} [Thu Sep 8 01:06:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:06:08.911922 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8ee1c-6350-480f-a4bf-a780f6cb240e None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:08.912928 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8ee1c-6350-480f-a4bf-a780f6cb240e None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:08.913416 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8ee1c-6350-480f-a4bf-a780f6cb240e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:08.958587 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=59 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:06:08.976953 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:06:09.008331 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-d4d8ee1c-6350-480f-a4bf-a780f6cb240e None None] MFA Rules not processed for user `301d30bbd3de4cc69e96bb12729442c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:06:09.010047 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-d4d8ee1c-6350-480f-a4bf-a780f6cb240e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:09.123929 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 665/1353] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Sep 8 01:06:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:06:10.515213 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58140f11-cee0-46f6-8c59-aa0e7d8849ba None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:10.515849 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58140f11-cee0-46f6-8c59-aa0e7d8849ba None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:10.515849 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58140f11-cee0-46f6-8c59-aa0e7d8849ba None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:10.517911 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 689/1354] 10.222.0.42 () {54 vars in 921 bytes} [Thu Sep 8 01:06:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:06:10.522915 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6001d2c-6245-4468-bf48-5f8b8c0b2f12 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:10.523338 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6001d2c-6245-4468-bf48-5f8b8c0b2f12 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:10.523587 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6001d2c-6245-4468-bf48-5f8b8c0b2f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:10.603423 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-e6001d2c-6245-4468-bf48-5f8b8c0b2f12 None None] MFA Rules not processed for user `301d30bbd3de4cc69e96bb12729442c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:06:10.604681 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-e6001d2c-6245-4468-bf48-5f8b8c0b2f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:10.651227 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:06:10.719807 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 666/1355] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Sep 8 01:06:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:06:14.248853 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890baf3-d43f-4818-8941-11bfa9b75eec None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:14.249747 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890baf3-d43f-4818-8941-11bfa9b75eec None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:14.250726 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890baf3-d43f-4818-8941-11bfa9b75eec None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:14.252412 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 690/1356] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:06:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:06:14.258574 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf17932-23ca-473c-8ecb-19284a5718ed None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:14.259067 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf17932-23ca-473c-8ecb-19284a5718ed None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:14.259644 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf17932-23ca-473c-8ecb-19284a5718ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:14.338809 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-9cf17932-23ca-473c-8ecb-19284a5718ed None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:06:14.340101 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-9cf17932-23ca-473c-8ecb-19284a5718ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:14.443317 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 667/1357] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:06:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:06:14.756935 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f86fd-1fc0-45c8-bdc3-e8652660f6bf None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:14.757489 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f86fd-1fc0-45c8-bdc3-e8652660f6bf None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:14.757971 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f86fd-1fc0-45c8-bdc3-e8652660f6bf None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:14.759553 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 691/1358] 10.222.0.42 () {54 vars in 962 bytes} [Thu Sep 8 01:06:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 01:06:14.768896 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f83b934-4387-43ed-934a-0a292aefea66 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:14.769269 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f83b934-4387-43ed-934a-0a292aefea66 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:14.769471 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f83b934-4387-43ed-934a-0a292aefea66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:14.786277 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:06:14.861086 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-6f83b934-4387-43ed-934a-0a292aefea66 None None] MFA Rules not processed for user `ae643cfdcb164aa9b474747cd21aa116`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:06:14.862419 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-6f83b934-4387-43ed-934a-0a292aefea66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:15.003520 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 668/1359] 10.222.0.42 () {58 vars in 1059 bytes} [Thu Sep 8 01:06:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:06:15.011034 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561f86fd-1fc0-45c8-bdc3-e8652660f6bf None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:06:15.013064 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-561f86fd-1fc0-45c8-bdc3-e8652660f6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:15.180955 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561f86fd-1fc0-45c8-bdc3-e8652660f6bf None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:06:15.264510 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:06:15.311488 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aabe79-a3b7-4e28-8fba-5e692aabaef7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae643cfdcb164aa9b474747cd21aa116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:06:15.312421 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aabe79-a3b7-4e28-8fba-5e692aabaef7 service nova] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:15.312813 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aabe79-a3b7-4e28-8fba-5e692aabaef7 service nova] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:15.313127 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aabe79-a3b7-4e28-8fba-5e692aabaef7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:15.314827 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-11aabe79-a3b7-4e28-8fba-5e692aabaef7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:15.427853 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aabe79-a3b7-4e28-8fba-5e692aabaef7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:06:15.429661 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aabe79-a3b7-4e28-8fba-5e692aabaef7 service nova] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:06:15.449541 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 692/1360] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:06:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 01:06:17.849456 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5b1d6-9674-4c5a-9e4e-ae0282c02f00 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:17.849456 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5b1d6-9674-4c5a-9e4e-ae0282c02f00 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:17.850824 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5b1d6-9674-4c5a-9e4e-ae0282c02f00 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:17.856416 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a41c6-a049-408d-96ec-f70ea40ef684 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:17.857127 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a41c6-a049-408d-96ec-f70ea40ef684 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:17.857127 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a41c6-a049-408d-96ec-f70ea40ef684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:17.860841 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 669/1361] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:06:17 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:06:17.926412 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-2e6a41c6-a049-408d-96ec-f70ea40ef684 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:06:17.927632 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-2e6a41c6-a049-408d-96ec-f70ea40ef684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:17.970009 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:06:18.035402 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 693/1362] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:06:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:06:18.330736 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b5b1d6-9674-4c5a-9e4e-ae0282c02f00 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:06:18.332308 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-c3b5b1d6-9674-4c5a-9e4e-ae0282c02f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:18.404838 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:06:18.510563 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b5b1d6-9674-4c5a-9e4e-ae0282c02f00 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:06:18.659668 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1af89f-f2a7-40e3-bc0c-0e319a0abf3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae643cfdcb164aa9b474747cd21aa116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:06:18.660334 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1af89f-f2a7-40e3-bc0c-0e319a0abf3e service nova] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:18.660611 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1af89f-f2a7-40e3-bc0c-0e319a0abf3e service nova] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:18.660879 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1af89f-f2a7-40e3-bc0c-0e319a0abf3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:18.662547 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-bc1af89f-f2a7-40e3-bc0c-0e319a0abf3e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:18.689314 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:06:18.758905 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1af89f-f2a7-40e3-bc0c-0e319a0abf3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:06:18.760643 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1af89f-f2a7-40e3-bc0c-0e319a0abf3e service nova] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:06:18.782372 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 670/1363] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:06:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:06:21.370890 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e606ce-c31d-451f-aa0d-61396ce61bd4 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:21.371426 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e606ce-c31d-451f-aa0d-61396ce61bd4 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:21.371426 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e606ce-c31d-451f-aa0d-61396ce61bd4 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:21.373001 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 694/1364] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:06:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:06:21.378476 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061643fe-f83f-4a78-80a8-27a7a9ff03bc None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:21.378942 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061643fe-f83f-4a78-80a8-27a7a9ff03bc None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:21.379032 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061643fe-f83f-4a78-80a8-27a7a9ff03bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:21.449769 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-061643fe-f83f-4a78-80a8-27a7a9ff03bc None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:06:21.450998 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-061643fe-f83f-4a78-80a8-27a7a9ff03bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:21.553189 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:06:21.564233 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 671/1365] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:06:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:06:21.846935 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d17936-af59-4ab5-971d-a9c84aefe919 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:21.847237 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d17936-af59-4ab5-971d-a9c84aefe919 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:21.847483 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d17936-af59-4ab5-971d-a9c84aefe919 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:21.849024 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 695/1366] 10.222.0.42 () {54 vars in 962 bytes} [Thu Sep 8 01:06:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:06:21.854449 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b10c04-b0f8-4f61-8663-0be74ff2cff6 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:21.854840 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b10c04-b0f8-4f61-8663-0be74ff2cff6 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:21.855218 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b10c04-b0f8-4f61-8663-0be74ff2cff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:21.935190 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-20b10c04-b0f8-4f61-8663-0be74ff2cff6 None None] MFA Rules not processed for user `ae643cfdcb164aa9b474747cd21aa116`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:06:21.936553 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-20b10c04-b0f8-4f61-8663-0be74ff2cff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:22.058461 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 672/1367] 10.222.0.42 () {58 vars in 1059 bytes} [Thu Sep 8 01:06:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:06:22.066590 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d17936-af59-4ab5-971d-a9c84aefe919 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:06:22.068254 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-20d17936-af59-4ab5-971d-a9c84aefe919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:22.224512 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d17936-af59-4ab5-971d-a9c84aefe919 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:06:22.246897 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:06:22.349939 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5b6347-75e8-4a69-a290-7e01537ce392 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae643cfdcb164aa9b474747cd21aa116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:06:22.350727 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b6347-75e8-4a69-a290-7e01537ce392 service nova] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:22.351111 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b6347-75e8-4a69-a290-7e01537ce392 service nova] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:22.351481 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b6347-75e8-4a69-a290-7e01537ce392 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:22.353235 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-9c5b6347-75e8-4a69-a290-7e01537ce392 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:22.459133 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5b6347-75e8-4a69-a290-7e01537ce392 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:06:22.461687 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5b6347-75e8-4a69-a290-7e01537ce392 service nova] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:06:22.488490 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 696/1368] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:06:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:06:22.573548 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15fca5b-24b0-436c-815d-e08bc0cb2893 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:22.574121 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15fca5b-24b0-436c-815d-e08bc0cb2893 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:22.574121 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15fca5b-24b0-436c-815d-e08bc0cb2893 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:22.575430 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 673/1369] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:06:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:06:22.580982 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b2251-f517-42b2-a4c9-cd0b0ae6722b None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:22.581268 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b2251-f517-42b2-a4c9-cd0b0ae6722b None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:22.581543 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b2251-f517-42b2-a4c9-cd0b0ae6722b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:22.624915 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:06:22.668693 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-f04b2251-f517-42b2-a4c9-cd0b0ae6722b None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:06:22.670092 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-f04b2251-f517-42b2-a4c9-cd0b0ae6722b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:22.789066 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 697/1370] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:06:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:06:23.122843 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d466f2-01ae-49ad-8d30-88ee20bb0a6c None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:23.123658 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d466f2-01ae-49ad-8d30-88ee20bb0a6c None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:23.124147 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d466f2-01ae-49ad-8d30-88ee20bb0a6c None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:23.125874 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 674/1371] 10.222.0.42 () {54 vars in 964 bytes} [Thu Sep 8 01:06:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:06:23.131060 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf0558-378a-4794-9ec4-42a70385dfe4 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:23.131339 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf0558-378a-4794-9ec4-42a70385dfe4 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:23.131613 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf0558-378a-4794-9ec4-42a70385dfe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:23.212953 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-f5bf0558-378a-4794-9ec4-42a70385dfe4 None None] MFA Rules not processed for user `a307e9150ead4877b970e09f36553968`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:06:23.214170 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-f5bf0558-378a-4794-9ec4-42a70385dfe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:23.266665 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:06:23.299341 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 698/1372] 10.222.0.42 () {58 vars in 1061 bytes} [Thu Sep 8 01:06:23 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:06:23.307132 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d466f2-01ae-49ad-8d30-88ee20bb0a6c None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:06:23.309019 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-99d466f2-01ae-49ad-8d30-88ee20bb0a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:23.364473 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:06:23.422997 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d466f2-01ae-49ad-8d30-88ee20bb0a6c None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:06:23.507226 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac80f20-3f91-48b8-845d-b29ef350fd04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:06:23.507846 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac80f20-3f91-48b8-845d-b29ef350fd04 service cinder] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:23.508126 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac80f20-3f91-48b8-845d-b29ef350fd04 service cinder] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:23.508444 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac80f20-3f91-48b8-845d-b29ef350fd04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:23.510071 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-5ac80f20-3f91-48b8-845d-b29ef350fd04 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:23.622100 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:06:23.650112 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac80f20-3f91-48b8-845d-b29ef350fd04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:06:23.653937 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac80f20-3f91-48b8-845d-b29ef350fd04 service cinder] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:06:23.680394 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 675/1373] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:06:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:06:24.782971 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107acb9-f58f-4ae9-a748-318740766d22 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:24.783536 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107acb9-f58f-4ae9-a748-318740766d22 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:24.783536 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107acb9-f58f-4ae9-a748-318740766d22 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:24.785019 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 699/1374] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:06:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:06:24.790430 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb1d85-0937-4fc0-b243-8afa42b4b039 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:24.790852 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb1d85-0937-4fc0-b243-8afa42b4b039 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:24.791128 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb1d85-0937-4fc0-b243-8afa42b4b039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:24.860863 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-7acb1d85-0937-4fc0-b243-8afa42b4b039 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:06:24.862093 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-7acb1d85-0937-4fc0-b243-8afa42b4b039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:24.958096 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 676/1375] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:06:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:06:25.252958 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9107acb9-f58f-4ae9-a748-318740766d22 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:06:25.254549 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-9107acb9-f58f-4ae9-a748-318740766d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:25.416886 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9107acb9-f58f-4ae9-a748-318740766d22 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:06:25.471295 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:06:25.539587 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1345a78e-bad0-4001-8879-3133f60be1ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae643cfdcb164aa9b474747cd21aa116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:06:25.540225 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1345a78e-bad0-4001-8879-3133f60be1ca service nova] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:25.540502 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1345a78e-bad0-4001-8879-3133f60be1ca service nova] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:25.540771 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1345a78e-bad0-4001-8879-3133f60be1ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:25.542366 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-1345a78e-bad0-4001-8879-3133f60be1ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:25.642499 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1345a78e-bad0-4001-8879-3133f60be1ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:06:25.644120 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1345a78e-bad0-4001-8879-3133f60be1ca service nova] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:06:25.663488 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 700/1376] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:06:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:06:28.098074 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a44a20-93f6-463d-baaa-46502ff60aed None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:28.098972 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a44a20-93f6-463d-baaa-46502ff60aed None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:28.099419 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a44a20-93f6-463d-baaa-46502ff60aed None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:28.101013 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 677/1377] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:06:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:06:28.106621 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6bc3f1-986c-4322-8b7d-c16697c15031 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:28.107066 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6bc3f1-986c-4322-8b7d-c16697c15031 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:28.107148 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6bc3f1-986c-4322-8b7d-c16697c15031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:28.177383 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-fc6bc3f1-986c-4322-8b7d-c16697c15031 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:06:28.178647 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-fc6bc3f1-986c-4322-8b7d-c16697c15031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:28.185992 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:06:28.288232 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 701/1378] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:06:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:06:28.578974 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a44a20-93f6-463d-baaa-46502ff60aed None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:06:28.580752 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-41a44a20-93f6-463d-baaa-46502ff60aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:28.646547 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:06:28.737601 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a44a20-93f6-463d-baaa-46502ff60aed None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:06:28.863225 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58847053-86b8-40ef-987e-6a29d1efd46b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae643cfdcb164aa9b474747cd21aa116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:06:28.863848 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58847053-86b8-40ef-987e-6a29d1efd46b service nova] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:28.864128 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58847053-86b8-40ef-987e-6a29d1efd46b service nova] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:28.864420 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58847053-86b8-40ef-987e-6a29d1efd46b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:28.866334 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-58847053-86b8-40ef-987e-6a29d1efd46b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:28.888122 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:06:28.969686 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58847053-86b8-40ef-987e-6a29d1efd46b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:06:28.971503 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58847053-86b8-40ef-987e-6a29d1efd46b service nova] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:06:28.991691 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 678/1379] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:06:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:06:31.267961 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e69318c-9267-45a2-a646-277aba022493 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:31.268415 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e69318c-9267-45a2-a646-277aba022493 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:31.268694 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e69318c-9267-45a2-a646-277aba022493 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:31.275235 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9af462-022b-488c-abac-60b88ea4097f None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:31.275636 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9af462-022b-488c-abac-60b88ea4097f None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:31.275904 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9af462-022b-488c-abac-60b88ea4097f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:31.280248 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 702/1380] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:06:31 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:06:31.393876 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-5b9af462-022b-488c-abac-60b88ea4097f None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:06:31.396163 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-5b9af462-022b-488c-abac-60b88ea4097f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:31.495189 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:06:31.501657 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 679/1381] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:06:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:06:31.791726 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e69318c-9267-45a2-a646-277aba022493 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:06:31.793369 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-5e69318c-9267-45a2-a646-277aba022493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:31.933038 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:06:31.944101 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e69318c-9267-45a2-a646-277aba022493 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:06:32.067429 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c983348-b08b-4e8a-9a21-0d48ee127658 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae643cfdcb164aa9b474747cd21aa116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:06:32.068029 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c983348-b08b-4e8a-9a21-0d48ee127658 service nova] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:32.068334 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c983348-b08b-4e8a-9a21-0d48ee127658 service nova] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:32.068605 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c983348-b08b-4e8a-9a21-0d48ee127658 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:32.070222 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-8c983348-b08b-4e8a-9a21-0d48ee127658 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:32.168719 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c983348-b08b-4e8a-9a21-0d48ee127658 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:06:32.170901 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c983348-b08b-4e8a-9a21-0d48ee127658 service nova] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:06:32.186269 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:06:32.196798 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 703/1382] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:06:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:06:35.214490 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9082301-ea65-47e7-9bfe-ed813fb93596 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:35.215224 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9082301-ea65-47e7-9bfe-ed813fb93596 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:35.215224 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9082301-ea65-47e7-9bfe-ed813fb93596 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:35.216485 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 680/1383] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:06:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:06:35.222931 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a458bd0c-eee7-4b21-b792-0fe25bffdccb None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:35.223613 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a458bd0c-eee7-4b21-b792-0fe25bffdccb None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:35.223613 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a458bd0c-eee7-4b21-b792-0fe25bffdccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:35.292350 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-a458bd0c-eee7-4b21-b792-0fe25bffdccb None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:06:35.293738 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-a458bd0c-eee7-4b21-b792-0fe25bffdccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:35.401326 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 704/1384] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:06:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:06:35.703059 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9082301-ea65-47e7-9bfe-ed813fb93596 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:06:35.705087 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-a9082301-ea65-47e7-9bfe-ed813fb93596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:35.884601 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9082301-ea65-47e7-9bfe-ed813fb93596 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:06:35.955383 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:06:36.003540 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f43ce4-3e18-4c96-b225-0044f333ad3b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae643cfdcb164aa9b474747cd21aa116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:06:36.004164 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f43ce4-3e18-4c96-b225-0044f333ad3b service nova] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:36.004447 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f43ce4-3e18-4c96-b225-0044f333ad3b service nova] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:36.004711 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f43ce4-3e18-4c96-b225-0044f333ad3b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:36.006305 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-e8f43ce4-3e18-4c96-b225-0044f333ad3b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:36.106479 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f43ce4-3e18-4c96-b225-0044f333ad3b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:06:36.108118 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f43ce4-3e18-4c96-b225-0044f333ad3b service nova] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:06:36.127025 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 681/1385] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:06:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:06:38.327680 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a65fc-af2e-48c5-ac61-66e6ab7809a3 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:38.328250 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a65fc-af2e-48c5-ac61-66e6ab7809a3 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:38.328250 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a65fc-af2e-48c5-ac61-66e6ab7809a3 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:38.329687 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 705/1386] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:06:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:06:38.336992 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae32bf-3c85-4d37-869e-dab8a031428d None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:38.337628 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae32bf-3c85-4d37-869e-dab8a031428d None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:38.337628 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae32bf-3c85-4d37-869e-dab8a031428d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:38.406465 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-70ae32bf-3c85-4d37-869e-dab8a031428d None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:06:38.407695 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-70ae32bf-3c85-4d37-869e-dab8a031428d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:38.441631 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:06:38.508936 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 682/1387] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:06:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:06:38.800364 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99a65fc-af2e-48c5-ac61-66e6ab7809a3 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:06:38.801890 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-e99a65fc-af2e-48c5-ac61-66e6ab7809a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:38.880910 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:06:38.952158 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99a65fc-af2e-48c5-ac61-66e6ab7809a3 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:06:39.068606 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f8a45-a508-425c-be8e-79402b60584b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae643cfdcb164aa9b474747cd21aa116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:06:39.069208 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f8a45-a508-425c-be8e-79402b60584b service nova] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:39.069481 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f8a45-a508-425c-be8e-79402b60584b service nova] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:39.069748 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f8a45-a508-425c-be8e-79402b60584b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:39.071317 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-9e7f8a45-a508-425c-be8e-79402b60584b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:39.112366 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:06:39.165953 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f8a45-a508-425c-be8e-79402b60584b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:06:39.167688 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f8a45-a508-425c-be8e-79402b60584b service nova] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:06:39.186204 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 706/1388] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:06:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:06:40.536923 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46d55ce-74e4-4db6-b72d-bd98b3b8683a None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:40.537491 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46d55ce-74e4-4db6-b72d-bd98b3b8683a None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:40.537491 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46d55ce-74e4-4db6-b72d-bd98b3b8683a None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:40.538866 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 683/1389] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:06:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:06:40.544358 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9869fcd7-6218-4aa0-8209-19a440a6eeaf None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:40.544766 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9869fcd7-6218-4aa0-8209-19a440a6eeaf None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:40.544988 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9869fcd7-6218-4aa0-8209-19a440a6eeaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:40.614688 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-9869fcd7-6218-4aa0-8209-19a440a6eeaf None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:06:40.615924 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-9869fcd7-6218-4aa0-8209-19a440a6eeaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:40.726037 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 707/1390] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:06:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:06:41.028537 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1462ead-09d7-4aaa-b714-7c04766921c8 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:41.028822 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1462ead-09d7-4aaa-b714-7c04766921c8 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:41.029125 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1462ead-09d7-4aaa-b714-7c04766921c8 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:41.030674 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 684/1391] 10.222.0.42 () {54 vars in 961 bytes} [Thu Sep 8 01:06:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:06:41.036064 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f9c42-b93e-48fe-ba7e-d0ee89b31c1f None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:41.036477 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f9c42-b93e-48fe-ba7e-d0ee89b31c1f None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:41.036818 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f9c42-b93e-48fe-ba7e-d0ee89b31c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:41.052961 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:06:41.115923 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-945f9c42-b93e-48fe-ba7e-d0ee89b31c1f None None] MFA Rules not processed for user `d1acfa7a09d34e2a9045c3f3fbea0f52`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:06:41.117080 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-945f9c42-b93e-48fe-ba7e-d0ee89b31c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:41.195983 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 708/1392] 10.222.0.42 () {58 vars in 1058 bytes} [Thu Sep 8 01:06:41 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:06:41.205229 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1462ead-09d7-4aaa-b714-7c04766921c8 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:06:41.206826 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-e1462ead-09d7-4aaa-b714-7c04766921c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:41.324670 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1462ead-09d7-4aaa-b714-7c04766921c8 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:06:41.385451 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:06:41.388456 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c94dfd-452e-446c-a33f-2cdff0dc29ef None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:41.388748 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c94dfd-452e-446c-a33f-2cdff0dc29ef None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:41.389023 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c94dfd-452e-446c-a33f-2cdff0dc29ef None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:41.390830 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 709/1393] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:06:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 01:06:41.396214 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797fe439-12d9-478a-a00d-df10cb0354ab None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:41.396503 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797fe439-12d9-478a-a00d-df10cb0354ab None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:41.396774 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797fe439-12d9-478a-a00d-df10cb0354ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:41.402840 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2139433b-c232-4e8a-a024-9d04c5135bfa service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1acfa7a09d34e2a9045c3f3fbea0f52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:06:41.403450 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2139433b-c232-4e8a-a024-9d04c5135bfa service glance] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:41.403734 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2139433b-c232-4e8a-a024-9d04c5135bfa service glance] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:41.404005 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2139433b-c232-4e8a-a024-9d04c5135bfa service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:41.405713 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-2139433b-c232-4e8a-a024-9d04c5135bfa service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:41.465515 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-797fe439-12d9-478a-a00d-df10cb0354ab None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:06:41.466767 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-797fe439-12d9-478a-a00d-df10cb0354ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:41.521420 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2139433b-c232-4e8a-a024-9d04c5135bfa service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:06:41.522864 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2139433b-c232-4e8a-a024-9d04c5135bfa service glance] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:06:41.525035 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:06:41.541919 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 685/1394] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:06:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:06:41.569065 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 710/1395] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:06:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:06:41.862783 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2139433b-c232-4e8a-a024-9d04c5135bfa service glance] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:06:41.863991 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-2139433b-c232-4e8a-a024-9d04c5135bfa service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:41.969534 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:06:41.978431 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2139433b-c232-4e8a-a024-9d04c5135bfa service glance] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:06:42.103948 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008d7175-7918-43e1-9351-c96bf764ec53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae643cfdcb164aa9b474747cd21aa116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:06:42.104714 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d7175-7918-43e1-9351-c96bf764ec53 service nova] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:42.105056 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d7175-7918-43e1-9351-c96bf764ec53 service nova] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:42.105394 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d7175-7918-43e1-9351-c96bf764ec53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:42.107160 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-008d7175-7918-43e1-9351-c96bf764ec53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:42.218648 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008d7175-7918-43e1-9351-c96bf764ec53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:06:42.220739 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008d7175-7918-43e1-9351-c96bf764ec53 service nova] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:06:42.241304 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:06:42.245292 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 686/1396] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:06:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:06:42.313826 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec73eb01-5ed0-42c3-8a30-74265b945188 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:42.314115 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec73eb01-5ed0-42c3-8a30-74265b945188 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:42.314392 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec73eb01-5ed0-42c3-8a30-74265b945188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:42.394471 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-ec73eb01-5ed0-42c3-8a30-74265b945188 None None] MFA Rules not processed for user `de75fd7d7721433684cafa2e6d3faaae`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:06:42.395737 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-ec73eb01-5ed0-42c3-8a30-74265b945188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:42.494031 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 711/1397] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Sep 8 01:06:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:06:42.536906 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19fef71-74b5-4c2f-afe3-e2b928aae455 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:42.537376 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19fef71-74b5-4c2f-afe3-e2b928aae455 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:42.537714 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19fef71-74b5-4c2f-afe3-e2b928aae455 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:42.539172 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 687/1398] 10.222.0.42 () {54 vars in 943 bytes} [Thu Sep 8 01:06:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:06:42.545685 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763fc35f-97ab-42a6-9c6a-568761b55ad5 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:42.545985 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763fc35f-97ab-42a6-9c6a-568761b55ad5 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:42.546258 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763fc35f-97ab-42a6-9c6a-568761b55ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:42.605039 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:06:42.625638 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-763fc35f-97ab-42a6-9c6a-568761b55ad5 None None] MFA Rules not processed for user `8add26872fe6479aa9d7de155a17195a`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:06:42.626912 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-763fc35f-97ab-42a6-9c6a-568761b55ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:42.738488 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 712/1399] 10.222.0.42 () {58 vars in 1040 bytes} [Thu Sep 8 01:06:42 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:06:42.747659 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19fef71-74b5-4c2f-afe3-e2b928aae455 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:06:42.750123 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-c19fef71-74b5-4c2f-afe3-e2b928aae455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:42.887146 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19fef71-74b5-4c2f-afe3-e2b928aae455 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:06:42.973645 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:06:42.993481 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b4a8fb-fd6d-43df-a122-98e41214d976 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8add26872fe6479aa9d7de155a17195a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:06:42.994095 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4a8fb-fd6d-43df-a122-98e41214d976 service swift] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:42.994390 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4a8fb-fd6d-43df-a122-98e41214d976 service swift] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:42.994693 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4a8fb-fd6d-43df-a122-98e41214d976 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:42.996305 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-c5b4a8fb-fd6d-43df-a122-98e41214d976 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:43.080431 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b4a8fb-fd6d-43df-a122-98e41214d976 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:06:43.082036 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b4a8fb-fd6d-43df-a122-98e41214d976 service swift] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:06:43.090197 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 688/1400] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Sep 8 01:06:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 01:06:44.482236 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dec361-3685-4486-8682-7247c152ec15 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:44.482861 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dec361-3685-4486-8682-7247c152ec15 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:44.482861 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dec361-3685-4486-8682-7247c152ec15 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:44.484620 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 713/1401] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:06:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:06:44.491477 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce27ed31-063a-4e5b-8a06-7c349577badd None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:44.491975 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce27ed31-063a-4e5b-8a06-7c349577badd None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:44.492092 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce27ed31-063a-4e5b-8a06-7c349577badd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:44.569606 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-ce27ed31-063a-4e5b-8a06-7c349577badd None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:06:44.570855 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-ce27ed31-063a-4e5b-8a06-7c349577badd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:44.656241 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:06:44.711600 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 689/1402] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:06:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:06:45.006622 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dec361-3685-4486-8682-7247c152ec15 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:06:45.008192 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-92dec361-3685-4486-8682-7247c152ec15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:45.126183 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:06:45.174114 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dec361-3685-4486-8682-7247c152ec15 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:06:45.305252 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aab37a-8130-4920-beb4-f5b89135d5cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae643cfdcb164aa9b474747cd21aa116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:06:45.305973 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aab37a-8130-4920-beb4-f5b89135d5cf service nova] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:45.306246 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aab37a-8130-4920-beb4-f5b89135d5cf service nova] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:45.306517 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aab37a-8130-4920-beb4-f5b89135d5cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:45.308283 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-f0aab37a-8130-4920-beb4-f5b89135d5cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:45.385769 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:06:45.413409 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aab37a-8130-4920-beb4-f5b89135d5cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:06:45.415271 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aab37a-8130-4920-beb4-f5b89135d5cf service nova] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:06:45.449912 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 714/1403] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:06:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:06:47.666627 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e5909-9f81-4f33-a35e-259c291395c1 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:47.667214 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e5909-9f81-4f33-a35e-259c291395c1 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:47.667214 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e5909-9f81-4f33-a35e-259c291395c1 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:47.668551 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 690/1404] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:06:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:06:47.674835 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aefb39-0e78-492a-a758-43144f1672fd None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:47.675279 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aefb39-0e78-492a-a758-43144f1672fd None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:47.675437 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aefb39-0e78-492a-a758-43144f1672fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:47.745267 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-39aefb39-0e78-492a-a758-43144f1672fd None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:06:47.746494 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-39aefb39-0e78-492a-a758-43144f1672fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:47.856035 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 715/1405] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:06:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:06:48.151491 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60e5909-9f81-4f33-a35e-259c291395c1 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:06:48.153061 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-a60e5909-9f81-4f33-a35e-259c291395c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:48.322812 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60e5909-9f81-4f33-a35e-259c291395c1 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:06:48.387471 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:06:48.453836 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496dcb44-5906-4b86-b808-b2d667b7dc86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae643cfdcb164aa9b474747cd21aa116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:06:48.454450 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496dcb44-5906-4b86-b808-b2d667b7dc86 service nova] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:48.454730 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496dcb44-5906-4b86-b808-b2d667b7dc86 service nova] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:48.455005 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496dcb44-5906-4b86-b808-b2d667b7dc86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:48.456718 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-496dcb44-5906-4b86-b808-b2d667b7dc86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:48.558367 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496dcb44-5906-4b86-b808-b2d667b7dc86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:06:48.560170 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496dcb44-5906-4b86-b808-b2d667b7dc86 service nova] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:06:48.580800 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 691/1406] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:06:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:06:50.834726 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e6884-3581-4687-ba77-2c352c096e35 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:50.835301 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e6884-3581-4687-ba77-2c352c096e35 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:50.835301 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e6884-3581-4687-ba77-2c352c096e35 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:50.836946 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 716/1407] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:06:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:06:50.842334 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec00515-9584-4fa0-bb65-45da1974daf4 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:50.842725 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec00515-9584-4fa0-bb65-45da1974daf4 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:50.842725 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec00515-9584-4fa0-bb65-45da1974daf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:50.914351 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-1ec00515-9584-4fa0-bb65-45da1974daf4 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:06:50.915504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-1ec00515-9584-4fa0-bb65-45da1974daf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:50.930414 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:06:51.024318 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 692/1408] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:06:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:06:51.310422 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460e6884-3581-4687-ba77-2c352c096e35 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:06:51.312623 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-460e6884-3581-4687-ba77-2c352c096e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:51.366946 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:06:51.476801 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460e6884-3581-4687-ba77-2c352c096e35 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:06:51.608645 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942f53bb-6bf2-4996-b5ac-56d3eff66281 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae643cfdcb164aa9b474747cd21aa116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:06:51.609818 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942f53bb-6bf2-4996-b5ac-56d3eff66281 service nova] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:06:51.610268 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942f53bb-6bf2-4996-b5ac-56d3eff66281 service nova] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:06:51.610543 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942f53bb-6bf2-4996-b5ac-56d3eff66281 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:06:51.612194 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-942f53bb-6bf2-4996-b5ac-56d3eff66281 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:06:51.625550 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:06:51.716980 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942f53bb-6bf2-4996-b5ac-56d3eff66281 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:06:51.719193 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942f53bb-6bf2-4996-b5ac-56d3eff66281 service nova] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:06:51.739495 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 717/1409] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:06:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:07:01.018980 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=37 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:07:01.737146 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=41 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:07:05.895229 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec4a79-3a04-4434-8550-71a164323519 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:05.895771 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec4a79-3a04-4434-8550-71a164323519 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:05.895771 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec4a79-3a04-4434-8550-71a164323519 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:05.897056 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 693/1410] 10.222.0.42 () {54 vars in 918 bytes} [Thu Sep 8 01:07:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:07:05.902028 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ed26a-5e1c-494c-a9de-dbe35801e5ef None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:05.902428 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ed26a-5e1c-494c-a9de-dbe35801e5ef None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:05.902578 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ed26a-5e1c-494c-a9de-dbe35801e5ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:05.978033 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-857ed26a-5e1c-494c-a9de-dbe35801e5ef None None] MFA Rules not processed for user `301d30bbd3de4cc69e96bb12729442c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:07:05.979244 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-857ed26a-5e1c-494c-a9de-dbe35801e5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:06.085494 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 718/1411] 10.222.0.42 () {58 vars in 1015 bytes} [Thu Sep 8 01:07:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:07:06.100856 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dae102-2b18-433a-914c-6313dec46285 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:06.101142 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dae102-2b18-433a-914c-6313dec46285 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:06.101418 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dae102-2b18-433a-914c-6313dec46285 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:06.102744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 694/1412] 10.222.0.42 () {54 vars in 918 bytes} [Thu Sep 8 01:07:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:07:06.107895 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c51287-2b09-4bff-9596-8275e1f1a128 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:06.108207 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c51287-2b09-4bff-9596-8275e1f1a128 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:06.108481 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c51287-2b09-4bff-9596-8275e1f1a128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:06.181299 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:07:06.184625 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-a9c51287-2b09-4bff-9596-8275e1f1a128 None None] MFA Rules not processed for user `ae643cfdcb164aa9b474747cd21aa116`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:07:06.187216 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-a9c51287-2b09-4bff-9596-8275e1f1a128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:06.302955 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 719/1413] 10.222.0.42 () {58 vars in 1015 bytes} [Thu Sep 8 01:07:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:07:08.881277 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addfbbd7-d07b-4283-ae31-52659663e28b None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:08.881747 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addfbbd7-d07b-4283-ae31-52659663e28b None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:08.881859 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addfbbd7-d07b-4283-ae31-52659663e28b None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:08.883200 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 695/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 01:07:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:07:08.889625 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab73043-b0d9-44c4-9daf-8137345920e6 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:08.890036 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab73043-b0d9-44c4-9daf-8137345920e6 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:08.890175 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab73043-b0d9-44c4-9daf-8137345920e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:08.957315 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-7ab73043-b0d9-44c4-9daf-8137345920e6 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:07:08.958527 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-7ab73043-b0d9-44c4-9daf-8137345920e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:09.013067 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:07:09.054962 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 720/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 01:07:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:07:09.351020 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addfbbd7-d07b-4283-ae31-52659663e28b None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:07:09.352599 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-addfbbd7-d07b-4283-ae31-52659663e28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:09.496972 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addfbbd7-d07b-4283-ae31-52659663e28b None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:07:09.568601 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:07:09.609126 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d348f8-a029-48e2-857e-4e4cc5870054 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae643cfdcb164aa9b474747cd21aa116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:07:09.609651 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d348f8-a029-48e2-857e-4e4cc5870054 service nova] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:09.609924 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d348f8-a029-48e2-857e-4e4cc5870054 service nova] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:09.610190 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d348f8-a029-48e2-857e-4e4cc5870054 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:09.611770 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-44d348f8-a029-48e2-857e-4e4cc5870054 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:09.699795 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d348f8-a029-48e2-857e-4e4cc5870054 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:07:09.701707 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d348f8-a029-48e2-857e-4e4cc5870054 service nova] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:07:09.719997 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 696/1416] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:07:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:07:10.865666 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ba9663-af05-4793-92e5-29c1646dab2d None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:10.866090 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ba9663-af05-4793-92e5-29c1646dab2d None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:10.866196 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ba9663-af05-4793-92e5-29c1646dab2d None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:10.867553 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 721/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 01:07:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:07:10.874371 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb1566-fc19-493e-8b2c-736f01cf82ed None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:10.874816 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb1566-fc19-493e-8b2c-736f01cf82ed None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:10.874923 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb1566-fc19-493e-8b2c-736f01cf82ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:10.950427 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-b2eb1566-fc19-493e-8b2c-736f01cf82ed None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:07:10.951686 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-b2eb1566-fc19-493e-8b2c-736f01cf82ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:10.978157 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:07:11.049746 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 697/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 01:07:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:07:11.199975 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ba9663-af05-4793-92e5-29c1646dab2d None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:07:11.202003 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-f1ba9663-af05-4793-92e5-29c1646dab2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:11.349092 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ba9663-af05-4793-92e5-29c1646dab2d None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:07:11.390462 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:07:11.464169 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9100cf36-e127-478d-b9cb-db8eb2e7b149 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae643cfdcb164aa9b474747cd21aa116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:07:11.464812 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100cf36-e127-478d-b9cb-db8eb2e7b149 service nova] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:11.465097 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100cf36-e127-478d-b9cb-db8eb2e7b149 service nova] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:11.465365 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100cf36-e127-478d-b9cb-db8eb2e7b149 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:11.467005 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-9100cf36-e127-478d-b9cb-db8eb2e7b149 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:11.557440 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9100cf36-e127-478d-b9cb-db8eb2e7b149 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:07:11.559037 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9100cf36-e127-478d-b9cb-db8eb2e7b149 service nova] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:07:11.577323 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 722/1419] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:07:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:07:12.719235 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e1a015-4dd2-4ede-867d-b838f5c3c7fa None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:12.719747 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e1a015-4dd2-4ede-867d-b838f5c3c7fa None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:12.719873 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e1a015-4dd2-4ede-867d-b838f5c3c7fa None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:12.721176 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 698/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 01:07:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:07:12.727258 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2493038f-33c1-4eac-9fe6-d324323ccc62 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:12.727735 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2493038f-33c1-4eac-9fe6-d324323ccc62 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:12.727841 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2493038f-33c1-4eac-9fe6-d324323ccc62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:12.781575 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:07:12.803688 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-2493038f-33c1-4eac-9fe6-d324323ccc62 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:07:12.804961 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-2493038f-33c1-4eac-9fe6-d324323ccc62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:12.903022 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 723/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 01:07:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:07:13.054665 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e1a015-4dd2-4ede-867d-b838f5c3c7fa None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:07:13.056256 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-45e1a015-4dd2-4ede-867d-b838f5c3c7fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:13.206914 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e1a015-4dd2-4ede-867d-b838f5c3c7fa None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:07:13.220047 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:07:13.322211 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f8af90-dcef-437b-b2ab-013020d8e5ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae643cfdcb164aa9b474747cd21aa116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:07:13.322817 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f8af90-dcef-437b-b2ab-013020d8e5ff service nova] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:13.323094 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f8af90-dcef-437b-b2ab-013020d8e5ff service nova] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:13.323365 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f8af90-dcef-437b-b2ab-013020d8e5ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:13.325007 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-d2f8af90-dcef-437b-b2ab-013020d8e5ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:13.413867 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f8af90-dcef-437b-b2ab-013020d8e5ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:07:13.415471 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f8af90-dcef-437b-b2ab-013020d8e5ff service nova] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:07:13.433240 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 699/1422] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:07:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:07:14.554600 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c6fd3-a7cf-46f3-9c43-a1c01939cc28 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:14.555164 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c6fd3-a7cf-46f3-9c43-a1c01939cc28 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:14.555164 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c6fd3-a7cf-46f3-9c43-a1c01939cc28 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:14.556611 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 724/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 01:07:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:07:14.563287 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5502a409-ea18-4b8e-8dd9-f501dde46a4e None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:14.563687 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5502a409-ea18-4b8e-8dd9-f501dde46a4e None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:14.563838 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5502a409-ea18-4b8e-8dd9-f501dde46a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:14.583175 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:07:14.641523 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-5502a409-ea18-4b8e-8dd9-f501dde46a4e None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:07:14.642742 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-5502a409-ea18-4b8e-8dd9-f501dde46a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:14.738181 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 700/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 01:07:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:07:14.888911 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4c6fd3-a7cf-46f3-9c43-a1c01939cc28 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:07:14.890465 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-ff4c6fd3-a7cf-46f3-9c43-a1c01939cc28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:14.999201 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:07:15.035663 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4c6fd3-a7cf-46f3-9c43-a1c01939cc28 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:07:15.148293 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6336b0a-e1db-4a71-9d45-b7346b4d901e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae643cfdcb164aa9b474747cd21aa116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:07:15.148824 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6336b0a-e1db-4a71-9d45-b7346b4d901e service nova] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:15.149104 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6336b0a-e1db-4a71-9d45-b7346b4d901e service nova] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:15.149379 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6336b0a-e1db-4a71-9d45-b7346b4d901e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:15.151101 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-d6336b0a-e1db-4a71-9d45-b7346b4d901e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:15.232412 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:07:15.246051 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6336b0a-e1db-4a71-9d45-b7346b4d901e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:07:15.247658 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6336b0a-e1db-4a71-9d45-b7346b4d901e service nova] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:07:15.266046 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 725/1425] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:07:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:07:16.384747 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6436d4-20c8-4fc6-a4c9-321c80acb60a None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:16.385313 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6436d4-20c8-4fc6-a4c9-321c80acb60a None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:16.385313 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6436d4-20c8-4fc6-a4c9-321c80acb60a None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:16.386835 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 701/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 01:07:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:07:16.395868 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b866355-e9a0-4dc8-8aee-8a20113a5741 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:16.396388 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b866355-e9a0-4dc8-8aee-8a20113a5741 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:16.396489 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b866355-e9a0-4dc8-8aee-8a20113a5741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:16.472000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-1b866355-e9a0-4dc8-8aee-8a20113a5741 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:07:16.473256 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-1b866355-e9a0-4dc8-8aee-8a20113a5741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:16.568767 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 726/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 01:07:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:07:16.720106 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6436d4-20c8-4fc6-a4c9-321c80acb60a None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:07:16.721591 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-3d6436d4-20c8-4fc6-a4c9-321c80acb60a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:16.806845 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:07:16.868485 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6436d4-20c8-4fc6-a4c9-321c80acb60a None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:07:16.980448 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbcc492-6c97-4478-8a78-df1f03acc3a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae643cfdcb164aa9b474747cd21aa116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:07:16.980978 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbcc492-6c97-4478-8a78-df1f03acc3a5 service nova] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:16.981255 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbcc492-6c97-4478-8a78-df1f03acc3a5 service nova] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:16.982048 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbcc492-6c97-4478-8a78-df1f03acc3a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:16.983654 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-3cbcc492-6c97-4478-8a78-df1f03acc3a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:17.036000 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:07:17.079063 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbcc492-6c97-4478-8a78-df1f03acc3a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:07:17.080715 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbcc492-6c97-4478-8a78-df1f03acc3a5 service nova] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:07:17.099172 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 702/1428] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:07:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:07:18.151690 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d5ddc-20c7-4aee-92c1-c6f100d35aa7 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:18.152289 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d5ddc-20c7-4aee-92c1-c6f100d35aa7 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:18.152289 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d5ddc-20c7-4aee-92c1-c6f100d35aa7 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:18.153661 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 727/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 01:07:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:07:18.159789 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dda6ba-c13a-4169-8369-cf6871e5fbb0 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:18.160249 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dda6ba-c13a-4169-8369-cf6871e5fbb0 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:18.160356 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dda6ba-c13a-4169-8369-cf6871e5fbb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:18.238584 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-64dda6ba-c13a-4169-8369-cf6871e5fbb0 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:07:18.239800 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-64dda6ba-c13a-4169-8369-cf6871e5fbb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:18.335777 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 703/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 01:07:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:07:18.487483 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13d5ddc-20c7-4aee-92c1-c6f100d35aa7 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:07:18.488950 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-a13d5ddc-20c7-4aee-92c1-c6f100d35aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:18.540422 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:07:18.636991 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13d5ddc-20c7-4aee-92c1-c6f100d35aa7 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:07:18.749357 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eebd0f-01cb-42f5-964b-136cc0cb6096 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae643cfdcb164aa9b474747cd21aa116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:07:18.749955 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eebd0f-01cb-42f5-964b-136cc0cb6096 service nova] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:18.750227 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eebd0f-01cb-42f5-964b-136cc0cb6096 service nova] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:18.750492 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eebd0f-01cb-42f5-964b-136cc0cb6096 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:18.752071 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-c8eebd0f-01cb-42f5-964b-136cc0cb6096 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:18.759070 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:07:18.843144 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eebd0f-01cb-42f5-964b-136cc0cb6096 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:07:18.844768 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eebd0f-01cb-42f5-964b-136cc0cb6096 service nova] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:07:18.863220 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 728/1431] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:07:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:07:19.914383 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e8c71-4d86-4c92-843a-feb60a2d0c8e None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:19.914925 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e8c71-4d86-4c92-843a-feb60a2d0c8e None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:19.914925 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e8c71-4d86-4c92-843a-feb60a2d0c8e None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:19.916204 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 704/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 01:07:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:07:19.923168 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaaecd7-f136-45d2-bb5a-ebf8d87eb3cd None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:19.923647 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaaecd7-f136-45d2-bb5a-ebf8d87eb3cd None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:19.923762 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaaecd7-f136-45d2-bb5a-ebf8d87eb3cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:19.999455 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-0eaaecd7-f136-45d2-bb5a-ebf8d87eb3cd None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:07:20.000695 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-0eaaecd7-f136-45d2-bb5a-ebf8d87eb3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:20.068023 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:07:20.097541 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 729/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 01:07:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:07:20.252784 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23e8c71-4d86-4c92-843a-feb60a2d0c8e None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:07:20.254324 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-f23e8c71-4d86-4c92-843a-feb60a2d0c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:20.271969 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:07:20.399759 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23e8c71-4d86-4c92-843a-feb60a2d0c8e None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:07:20.489222 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:07:20.515668 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2d52d9-af9b-4315-b719-f1cd73cbfd90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae643cfdcb164aa9b474747cd21aa116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:07:20.516332 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d52d9-af9b-4315-b719-f1cd73cbfd90 service nova] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:20.516608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d52d9-af9b-4315-b719-f1cd73cbfd90 service nova] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:20.516875 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d52d9-af9b-4315-b719-f1cd73cbfd90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:20.518487 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-ac2d52d9-af9b-4315-b719-f1cd73cbfd90 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:20.608353 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2d52d9-af9b-4315-b719-f1cd73cbfd90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:07:20.609970 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2d52d9-af9b-4315-b719-f1cd73cbfd90 service nova] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:07:20.627639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 705/1434] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:07:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:07:21.805072 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830f09c-0bb8-4ccf-80bf-4c5f374a9771 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:21.805667 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830f09c-0bb8-4ccf-80bf-4c5f374a9771 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:21.805667 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830f09c-0bb8-4ccf-80bf-4c5f374a9771 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:21.806960 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 730/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 01:07:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 01:07:21.813665 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad484b39-dbb1-41af-acc3-dd374827dc96 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:21.814088 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad484b39-dbb1-41af-acc3-dd374827dc96 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:21.814202 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad484b39-dbb1-41af-acc3-dd374827dc96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:21.890518 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-ad484b39-dbb1-41af-acc3-dd374827dc96 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:07:21.891665 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-ad484b39-dbb1-41af-acc3-dd374827dc96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:21.933226 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:07:21.987068 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 706/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 01:07:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:07:22.139395 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6830f09c-0bb8-4ccf-80bf-4c5f374a9771 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:07:22.141051 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-6830f09c-0bb8-4ccf-80bf-4c5f374a9771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:22.288877 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6830f09c-0bb8-4ccf-80bf-4c5f374a9771 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:07:22.343880 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:07:22.404403 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7d725-1834-4c42-a77d-86f3a8a178eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae643cfdcb164aa9b474747cd21aa116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:07:22.404936 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7d725-1834-4c42-a77d-86f3a8a178eb service nova] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:22.405211 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7d725-1834-4c42-a77d-86f3a8a178eb service nova] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:22.405479 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7d725-1834-4c42-a77d-86f3a8a178eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:22.407101 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-b1c7d725-1834-4c42-a77d-86f3a8a178eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:22.495936 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c7d725-1834-4c42-a77d-86f3a8a178eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:07:22.497763 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c7d725-1834-4c42-a77d-86f3a8a178eb service nova] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:07:22.515753 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 731/1437] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:07:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:07:23.572988 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7f699-5b0c-4a28-8cf7-a1bc0e251b4b None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:23.573538 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7f699-5b0c-4a28-8cf7-a1bc0e251b4b None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:23.573538 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7f699-5b0c-4a28-8cf7-a1bc0e251b4b None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:23.574868 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 707/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 01:07:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:07:23.581923 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b6b77-8704-488c-9e40-2f0924bd95fa None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:23.582373 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b6b77-8704-488c-9e40-2f0924bd95fa None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:23.582469 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b6b77-8704-488c-9e40-2f0924bd95fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:23.659277 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-fb1b6b77-8704-488c-9e40-2f0924bd95fa None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:07:23.660519 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-fb1b6b77-8704-488c-9e40-2f0924bd95fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:23.662561 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:07:23.757000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 732/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 01:07:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:07:23.905405 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd7f699-5b0c-4a28-8cf7-a1bc0e251b4b None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:07:23.906927 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-8cd7f699-5b0c-4a28-8cf7-a1bc0e251b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:24.050106 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd7f699-5b0c-4a28-8cf7-a1bc0e251b4b None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:07:24.078023 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:07:24.163395 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08693564-bebe-483b-ba35-f6fef8f568be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae643cfdcb164aa9b474747cd21aa116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:07:24.163921 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08693564-bebe-483b-ba35-f6fef8f568be service nova] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:24.164294 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08693564-bebe-483b-ba35-f6fef8f568be service nova] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:24.164568 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08693564-bebe-483b-ba35-f6fef8f568be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:24.166193 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-08693564-bebe-483b-ba35-f6fef8f568be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:24.255009 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08693564-bebe-483b-ba35-f6fef8f568be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:07:24.256660 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08693564-bebe-483b-ba35-f6fef8f568be service nova] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:07:24.275253 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 708/1440] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:07:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 01:07:26.350261 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b18e925-734c-4d89-a3bd-e1722c49f6d3 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:26.350794 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b18e925-734c-4d89-a3bd-e1722c49f6d3 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:26.350794 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b18e925-734c-4d89-a3bd-e1722c49f6d3 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:26.352096 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 733/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 01:07:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:07:26.358378 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2452b317-d26a-4922-811c-cf6d242f9ac5 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:26.358822 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2452b317-d26a-4922-811c-cf6d242f9ac5 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:26.358921 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2452b317-d26a-4922-811c-cf6d242f9ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:26.399784 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:07:26.426466 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-2452b317-d26a-4922-811c-cf6d242f9ac5 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:07:26.427682 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-2452b317-d26a-4922-811c-cf6d242f9ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:26.524683 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 709/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 01:07:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:07:26.808798 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b18e925-734c-4d89-a3bd-e1722c49f6d3 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:07:26.810881 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-1b18e925-734c-4d89-a3bd-e1722c49f6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:26.914471 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b18e925-734c-4d89-a3bd-e1722c49f6d3 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:07:26.946699 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:07:26.989750 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f304492-d6a3-48e8-96ae-a58c4ad9d97b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:07:26.990345 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f304492-d6a3-48e8-96ae-a58c4ad9d97b service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:26.990620 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f304492-d6a3-48e8-96ae-a58c4ad9d97b service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:26.990889 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f304492-d6a3-48e8-96ae-a58c4ad9d97b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:26.992554 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-6f304492-d6a3-48e8-96ae-a58c4ad9d97b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:27.099878 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f304492-d6a3-48e8-96ae-a58c4ad9d97b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:07:27.101350 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f304492-d6a3-48e8-96ae-a58c4ad9d97b service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:07:27.119974 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 734/1443] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:07:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:07:29.532475 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c386c5-139e-4d58-941a-d7d223dabd37 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:29.533077 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c386c5-139e-4d58-941a-d7d223dabd37 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:29.533077 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c386c5-139e-4d58-941a-d7d223dabd37 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:29.534405 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 710/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 01:07:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:07:29.539708 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea3196d-38ba-4a3f-8441-7b634331b280 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:29.540102 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea3196d-38ba-4a3f-8441-7b634331b280 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:29.540286 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea3196d-38ba-4a3f-8441-7b634331b280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:29.606084 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-1ea3196d-38ba-4a3f-8441-7b634331b280 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:07:29.607332 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-1ea3196d-38ba-4a3f-8441-7b634331b280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:29.620942 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:07:29.706558 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 735/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 01:07:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:07:29.987629 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c386c5-139e-4d58-941a-d7d223dabd37 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:07:29.989309 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-f5c386c5-139e-4d58-941a-d7d223dabd37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:30.092750 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c386c5-139e-4d58-941a-d7d223dabd37 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:07:30.105322 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:07:30.167436 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f5a65c-187b-49ac-81a1-5cc22eafe1b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:07:30.168035 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5a65c-187b-49ac-81a1-5cc22eafe1b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:30.168343 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5a65c-187b-49ac-81a1-5cc22eafe1b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:30.169246 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5a65c-187b-49ac-81a1-5cc22eafe1b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:30.170839 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-e0f5a65c-187b-49ac-81a1-5cc22eafe1b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:30.282531 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f5a65c-187b-49ac-81a1-5cc22eafe1b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:07:30.283970 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f5a65c-187b-49ac-81a1-5cc22eafe1b7 service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:07:30.302016 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 711/1446] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:07:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:07:32.777442 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6c689-0701-4a4c-bbd3-1d69790f0ca6 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:32.778550 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6c689-0701-4a4c-bbd3-1d69790f0ca6 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:32.778768 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6c689-0701-4a4c-bbd3-1d69790f0ca6 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:32.780329 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 736/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 01:07:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:07:32.785644 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfef588-68bb-4236-8e16-bd144e773ea9 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:32.786085 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfef588-68bb-4236-8e16-bd144e773ea9 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:32.786191 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfef588-68bb-4236-8e16-bd144e773ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:32.834089 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:07:32.852999 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-6dfef588-68bb-4236-8e16-bd144e773ea9 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:07:32.854210 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-6dfef588-68bb-4236-8e16-bd144e773ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:32.950703 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 712/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 01:07:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:07:33.235041 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb6c689-0701-4a4c-bbd3-1d69790f0ca6 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:07:33.236618 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-ecb6c689-0701-4a4c-bbd3-1d69790f0ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:33.339333 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb6c689-0701-4a4c-bbd3-1d69790f0ca6 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:07:33.380077 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:07:33.413938 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0452a42-c9eb-4239-add1-6f28cda54deb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:07:33.414532 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0452a42-c9eb-4239-add1-6f28cda54deb service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:33.414804 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0452a42-c9eb-4239-add1-6f28cda54deb service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:33.415069 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0452a42-c9eb-4239-add1-6f28cda54deb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:33.416678 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-c0452a42-c9eb-4239-add1-6f28cda54deb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:33.525685 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0452a42-c9eb-4239-add1-6f28cda54deb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:07:33.527159 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0452a42-c9eb-4239-add1-6f28cda54deb service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:07:33.546409 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 737/1449] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:07:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:07:35.846305 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da6bb2-2f9c-4a39-8365-85f391b05964 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:35.846871 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da6bb2-2f9c-4a39-8365-85f391b05964 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:35.846871 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da6bb2-2f9c-4a39-8365-85f391b05964 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:35.848242 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 713/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 01:07:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:07:35.853734 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f8e91b-2b91-47b5-8fe7-e46d258835c0 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:35.854185 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f8e91b-2b91-47b5-8fe7-e46d258835c0 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:35.854283 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f8e91b-2b91-47b5-8fe7-e46d258835c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:35.921460 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-70f8e91b-2b91-47b5-8fe7-e46d258835c0 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:07:35.922600 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-70f8e91b-2b91-47b5-8fe7-e46d258835c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:35.943794 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:07:36.019959 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 738/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 01:07:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:07:36.308902 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87da6bb2-2f9c-4a39-8365-85f391b05964 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:07:36.310545 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-87da6bb2-2f9c-4a39-8365-85f391b05964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:36.414476 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87da6bb2-2f9c-4a39-8365-85f391b05964 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:07:36.434744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:07:36.488736 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cdbbdd-190b-423d-8ba9-9243b317a402 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:07:36.489335 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cdbbdd-190b-423d-8ba9-9243b317a402 service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:36.489610 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cdbbdd-190b-423d-8ba9-9243b317a402 service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:36.489877 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cdbbdd-190b-423d-8ba9-9243b317a402 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:36.491474 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-f7cdbbdd-190b-423d-8ba9-9243b317a402 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:36.600684 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cdbbdd-190b-423d-8ba9-9243b317a402 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:07:36.602046 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cdbbdd-190b-423d-8ba9-9243b317a402 service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:07:36.619792 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 714/1452] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:07:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:07:38.819123 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f32c2c3-28d5-49c8-b212-7de0efbd5b70 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:38.819684 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f32c2c3-28d5-49c8-b212-7de0efbd5b70 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:38.819684 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f32c2c3-28d5-49c8-b212-7de0efbd5b70 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:38.821049 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 739/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 01:07:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:07:38.827190 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c9365-38cf-4728-bba2-2b67eb3cd54c None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:38.827624 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c9365-38cf-4728-bba2-2b67eb3cd54c None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:38.827739 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c9365-38cf-4728-bba2-2b67eb3cd54c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:38.891491 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:07:38.894214 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-6d5c9365-38cf-4728-bba2-2b67eb3cd54c None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:07:38.896024 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-6d5c9365-38cf-4728-bba2-2b67eb3cd54c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:38.991156 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 715/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 01:07:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:07:39.277262 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f32c2c3-28d5-49c8-b212-7de0efbd5b70 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:07:39.279002 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-3f32c2c3-28d5-49c8-b212-7de0efbd5b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:39.387707 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f32c2c3-28d5-49c8-b212-7de0efbd5b70 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:07:39.438717 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:07:39.463272 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8192fa36-ede9-43d1-8a8e-a11d3164073a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:07:39.463876 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8192fa36-ede9-43d1-8a8e-a11d3164073a service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:39.464179 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8192fa36-ede9-43d1-8a8e-a11d3164073a service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:39.464455 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8192fa36-ede9-43d1-8a8e-a11d3164073a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:39.466123 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-8192fa36-ede9-43d1-8a8e-a11d3164073a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:39.577134 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8192fa36-ede9-43d1-8a8e-a11d3164073a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:07:39.578504 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8192fa36-ede9-43d1-8a8e-a11d3164073a service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:07:39.596747 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 740/1455] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:07:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:07:41.833714 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef31c3eb-d412-4828-8deb-09ad5319af15 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:41.834269 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef31c3eb-d412-4828-8deb-09ad5319af15 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:41.834269 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef31c3eb-d412-4828-8deb-09ad5319af15 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:41.835584 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 716/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 01:07:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:07:41.841862 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c7470d-9c6a-4ab6-83da-f3142999b96f None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:41.842273 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c7470d-9c6a-4ab6-83da-f3142999b96f None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:41.842399 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c7470d-9c6a-4ab6-83da-f3142999b96f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:41.908787 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-e7c7470d-9c6a-4ab6-83da-f3142999b96f None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:07:41.909937 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-e7c7470d-9c6a-4ab6-83da-f3142999b96f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:41.939786 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:07:42.009590 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 741/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 01:07:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:07:42.294806 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef31c3eb-d412-4828-8deb-09ad5319af15 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:07:42.296392 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-ef31c3eb-d412-4828-8deb-09ad5319af15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:42.398718 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef31c3eb-d412-4828-8deb-09ad5319af15 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:07:42.427273 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:07:42.472768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0867056-09a9-443c-a537-f2cd585e2b38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:07:42.473400 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0867056-09a9-443c-a537-f2cd585e2b38 service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:42.473702 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0867056-09a9-443c-a537-f2cd585e2b38 service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:42.473972 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0867056-09a9-443c-a537-f2cd585e2b38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:42.475573 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-c0867056-09a9-443c-a537-f2cd585e2b38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:42.583055 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0867056-09a9-443c-a537-f2cd585e2b38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:07:42.584441 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0867056-09a9-443c-a537-f2cd585e2b38 service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:07:42.602650 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 717/1458] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:07:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:07:44.831217 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5317f46-ff77-45ac-bb82-490ea7e8b577 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:44.831780 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5317f46-ff77-45ac-bb82-490ea7e8b577 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:44.831780 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5317f46-ff77-45ac-bb82-490ea7e8b577 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:44.833142 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 742/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 01:07:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:07:44.838340 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f720c-1e11-43a7-8dd9-6b26a9d8d9a2 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:44.838782 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f720c-1e11-43a7-8dd9-6b26a9d8d9a2 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:44.838891 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f720c-1e11-43a7-8dd9-6b26a9d8d9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:44.905326 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-7e1f720c-1e11-43a7-8dd9-6b26a9d8d9a2 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:07:44.906478 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-7e1f720c-1e11-43a7-8dd9-6b26a9d8d9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:44.914668 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:07:45.002230 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 718/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 01:07:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:07:45.285575 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5317f46-ff77-45ac-bb82-490ea7e8b577 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:07:45.287114 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-f5317f46-ff77-45ac-bb82-490ea7e8b577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:45.397191 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5317f46-ff77-45ac-bb82-490ea7e8b577 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:07:45.453726 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:07:45.471731 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868c3888-ca20-4f4f-a2f4-c0a196064649 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:07:45.472368 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c3888-ca20-4f4f-a2f4-c0a196064649 service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:45.472645 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c3888-ca20-4f4f-a2f4-c0a196064649 service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:45.472914 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c3888-ca20-4f4f-a2f4-c0a196064649 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:45.474522 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-868c3888-ca20-4f4f-a2f4-c0a196064649 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:45.584167 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868c3888-ca20-4f4f-a2f4-c0a196064649 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:07:45.585541 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868c3888-ca20-4f4f-a2f4-c0a196064649 service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:07:45.721019 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 743/1461] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:07:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:07:47.929974 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9059cc-cf90-4cd4-a3f0-04d3fd6eaa27 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:47.930538 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9059cc-cf90-4cd4-a3f0-04d3fd6eaa27 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:47.930538 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9059cc-cf90-4cd4-a3f0-04d3fd6eaa27 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:47.931802 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 719/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 01:07:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:07:47.937349 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1602be-3de0-45d5-8916-e2d41d6f4a55 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:47.937821 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1602be-3de0-45d5-8916-e2d41d6f4a55 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:47.937930 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1602be-3de0-45d5-8916-e2d41d6f4a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:48.002694 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-2b1602be-3de0-45d5-8916-e2d41d6f4a55 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:07:48.003903 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-2b1602be-3de0-45d5-8916-e2d41d6f4a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:48.042042 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:07:48.100667 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 744/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 01:07:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:07:48.382871 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9059cc-cf90-4cd4-a3f0-04d3fd6eaa27 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:07:48.384471 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-8c9059cc-cf90-4cd4-a3f0-04d3fd6eaa27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:48.487374 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9059cc-cf90-4cd4-a3f0-04d3fd6eaa27 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:07:48.525417 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:07:48.561951 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64531f36-18ee-48aa-85b4-5e4ed5246612 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:07:48.562550 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64531f36-18ee-48aa-85b4-5e4ed5246612 service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:48.562831 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64531f36-18ee-48aa-85b4-5e4ed5246612 service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:48.563102 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64531f36-18ee-48aa-85b4-5e4ed5246612 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:48.564731 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-64531f36-18ee-48aa-85b4-5e4ed5246612 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:48.675350 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64531f36-18ee-48aa-85b4-5e4ed5246612 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:07:48.676905 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64531f36-18ee-48aa-85b4-5e4ed5246612 service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:07:48.695254 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 720/1464] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:07:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:07:50.866595 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ff76e4-1175-4834-96ea-48c1df0d83cf None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:50.867184 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ff76e4-1175-4834-96ea-48c1df0d83cf None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:50.867184 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ff76e4-1175-4834-96ea-48c1df0d83cf None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:50.868544 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 745/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 01:07:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:07:50.874405 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a847d-d61b-4c1c-ace0-9e55f5b90149 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:50.874846 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a847d-d61b-4c1c-ace0-9e55f5b90149 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:50.874956 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a847d-d61b-4c1c-ace0-9e55f5b90149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:50.942230 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-fb1a847d-d61b-4c1c-ace0-9e55f5b90149 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:07:50.943869 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-fb1a847d-d61b-4c1c-ace0-9e55f5b90149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:50.963634 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:07:51.040881 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 721/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 01:07:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:07:51.327718 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ff76e4-1175-4834-96ea-48c1df0d83cf None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:07:51.329325 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-c9ff76e4-1175-4834-96ea-48c1df0d83cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:51.437472 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ff76e4-1175-4834-96ea-48c1df0d83cf None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:07:51.504794 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:07:51.512935 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ea4945-d6b5-4477-b743-93f7c0db5322 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:07:51.513537 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea4945-d6b5-4477-b743-93f7c0db5322 service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:51.513813 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea4945-d6b5-4477-b743-93f7c0db5322 service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:51.514084 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea4945-d6b5-4477-b743-93f7c0db5322 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:51.515657 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-f4ea4945-d6b5-4477-b743-93f7c0db5322 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:51.624732 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ea4945-d6b5-4477-b743-93f7c0db5322 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:07:51.626113 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ea4945-d6b5-4477-b743-93f7c0db5322 service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:07:51.644340 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 746/1467] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:07:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:07:53.825196 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005dcb4-dfaf-4dbc-8e04-d30925cd7cdb None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:53.825753 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005dcb4-dfaf-4dbc-8e04-d30925cd7cdb None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:53.825753 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005dcb4-dfaf-4dbc-8e04-d30925cd7cdb None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:53.827083 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 722/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 01:07:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:07:53.832277 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a7f5e-d511-458e-8beb-e98501480f60 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:53.832679 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a7f5e-d511-458e-8beb-e98501480f60 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:53.832820 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a7f5e-d511-458e-8beb-e98501480f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:53.898555 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-725a7f5e-d511-458e-8beb-e98501480f60 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:07:53.899668 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-725a7f5e-d511-458e-8beb-e98501480f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:53.947912 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:07:53.996844 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 747/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 01:07:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:07:54.278095 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5005dcb4-dfaf-4dbc-8e04-d30925cd7cdb None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:07:54.279640 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-5005dcb4-dfaf-4dbc-8e04-d30925cd7cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:54.382016 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5005dcb4-dfaf-4dbc-8e04-d30925cd7cdb None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:07:54.429858 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:07:54.455165 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9328bbb-5d0c-41d9-a290-cc65cdf132a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:07:54.455694 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9328bbb-5d0c-41d9-a290-cc65cdf132a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:07:54.455968 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9328bbb-5d0c-41d9-a290-cc65cdf132a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:07:54.456269 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9328bbb-5d0c-41d9-a290-cc65cdf132a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:07:54.457840 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-d9328bbb-5d0c-41d9-a290-cc65cdf132a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:07:54.566320 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9328bbb-5d0c-41d9-a290-cc65cdf132a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:07:54.567764 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9328bbb-5d0c-41d9-a290-cc65cdf132a7 service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:07:54.586571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 723/1470] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:07:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:08:03.995105 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=22 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:08:04.584325 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=66 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:08:14.539677 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c690398-8aa3-4fae-b63a-10ff519985b5 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:08:14.540284 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c690398-8aa3-4fae-b63a-10ff519985b5 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:08:14.540284 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c690398-8aa3-4fae-b63a-10ff519985b5 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:08:14.541819 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 748/1471] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:08:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:08:14.547218 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c006e-d2b7-45db-bd51-5b9aa012b87b None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:08:14.547647 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c006e-d2b7-45db-bd51-5b9aa012b87b None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:08:14.547756 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c006e-d2b7-45db-bd51-5b9aa012b87b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:08:14.613316 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-f73c006e-d2b7-45db-bd51-5b9aa012b87b None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:08:14.614547 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-f73c006e-d2b7-45db-bd51-5b9aa012b87b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:08:14.715288 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 724/1472] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:08:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:08:15.013485 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5acd0d-e207-4389-bf6b-dca1462b73e0 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:08:15.013760 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5acd0d-e207-4389-bf6b-dca1462b73e0 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:08:15.014029 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5acd0d-e207-4389-bf6b-dca1462b73e0 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:08:15.015346 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 749/1473] 10.222.0.42 () {54 vars in 961 bytes} [Thu Sep 8 01:08:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:08:15.020415 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f269bc2-32b9-43e4-815c-734015be9f0a None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:08:15.020700 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f269bc2-32b9-43e4-815c-734015be9f0a None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:08:15.020971 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f269bc2-32b9-43e4-815c-734015be9f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:08:15.096624 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-9f269bc2-32b9-43e4-815c-734015be9f0a None None] MFA Rules not processed for user `d1acfa7a09d34e2a9045c3f3fbea0f52`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:08:15.097823 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-9f269bc2-32b9-43e4-815c-734015be9f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:08:15.118254 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:08:15.174150 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 725/1474] 10.222.0.42 () {58 vars in 1058 bytes} [Thu Sep 8 01:08:15 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:08:15.181797 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5acd0d-e207-4389-bf6b-dca1462b73e0 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:08:15.183357 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-dd5acd0d-e207-4389-bf6b-dca1462b73e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:08:15.286550 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5acd0d-e207-4389-bf6b-dca1462b73e0 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:08:15.361275 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ee379-af03-4f16-a77c-4c5bbd433bc1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1acfa7a09d34e2a9045c3f3fbea0f52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:08:15.361877 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ee379-af03-4f16-a77c-4c5bbd433bc1 service glance] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:08:15.362154 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ee379-af03-4f16-a77c-4c5bbd433bc1 service glance] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:08:15.362423 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ee379-af03-4f16-a77c-4c5bbd433bc1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:08:15.364018 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-c01ee379-af03-4f16-a77c-4c5bbd433bc1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:08:15.419496 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:08:15.476857 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01ee379-af03-4f16-a77c-4c5bbd433bc1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:08:15.478309 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01ee379-af03-4f16-a77c-4c5bbd433bc1 service glance] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:08:15.497622 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 750/1475] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:08:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:08:17.740022 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cdb0b2-d6a1-4b61-abfc-4965e362b194 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:08:17.740677 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cdb0b2-d6a1-4b61-abfc-4965e362b194 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:08:17.740677 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cdb0b2-d6a1-4b61-abfc-4965e362b194 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:08:17.741933 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 726/1476] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:08:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:08:17.747710 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce576537-f983-41bd-b84e-4bcfc060d4eb None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:08:17.748208 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce576537-f983-41bd-b84e-4bcfc060d4eb None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:08:17.748317 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce576537-f983-41bd-b84e-4bcfc060d4eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:08:17.813649 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-ce576537-f983-41bd-b84e-4bcfc060d4eb None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:08:17.814844 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-ce576537-f983-41bd-b84e-4bcfc060d4eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:08:17.908965 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 751/1477] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:08:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:08:18.181020 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cdb0b2-d6a1-4b61-abfc-4965e362b194 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:08:18.182659 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-90cdb0b2-d6a1-4b61-abfc-4965e362b194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:08:18.309238 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cdb0b2-d6a1-4b61-abfc-4965e362b194 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:08:18.476863 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:08:18.521821 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ee0114-491a-458d-b4f9-3949416a258e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:08:18.522423 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee0114-491a-458d-b4f9-3949416a258e admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:08:18.522701 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee0114-491a-458d-b4f9-3949416a258e admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:08:18.522972 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee0114-491a-458d-b4f9-3949416a258e admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:08:18.523608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ee0114-491a-458d-b4f9-3949416a258e admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:08:18.525383 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: WARNING py.warnings [None req-f3ee0114-491a-458d-b4f9-3949416a258e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 01:08:18.525383 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: warnings.warn(msg) Sep 08 01:08:18.525383 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: Sep 08 01:08:18.526623 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ee0114-491a-458d-b4f9-3949416a258e admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:08:18.545377 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 727/1478] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Sep 8 01:08:18 2022] GET /identity/v3/projects => generated 2553 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:08:20.742828 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc273135-9698-4174-a539-76649a9a07f8 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:08:20.743475 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc273135-9698-4174-a539-76649a9a07f8 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:08:20.743475 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc273135-9698-4174-a539-76649a9a07f8 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:08:20.745540 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 752/1479] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:08:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 01:08:20.751095 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89db247-2850-4c37-8496-b0742a4f5b1b None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:08:20.751569 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89db247-2850-4c37-8496-b0742a4f5b1b None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:08:20.751686 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89db247-2850-4c37-8496-b0742a4f5b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:08:20.831953 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-d89db247-2850-4c37-8496-b0742a4f5b1b None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:08:20.833269 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-d89db247-2850-4c37-8496-b0742a4f5b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:08:20.968455 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 728/1480] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:08:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:08:21.320297 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc273135-9698-4174-a539-76649a9a07f8 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:08:21.321826 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-fc273135-9698-4174-a539-76649a9a07f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:08:21.332930 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:08:21.497989 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc273135-9698-4174-a539-76649a9a07f8 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:08:21.579677 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:08:21.612629 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2844f8-806d-41de-ac03-e59a895c05c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae643cfdcb164aa9b474747cd21aa116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:08:21.613230 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2844f8-806d-41de-ac03-e59a895c05c3 service nova] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:08:21.613505 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2844f8-806d-41de-ac03-e59a895c05c3 service nova] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:08:21.613773 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2844f8-806d-41de-ac03-e59a895c05c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:08:21.615360 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-7f2844f8-806d-41de-ac03-e59a895c05c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:08:21.707079 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2844f8-806d-41de-ac03-e59a895c05c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:08:21.708709 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2844f8-806d-41de-ac03-e59a895c05c3 service nova] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:08:21.726605 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 753/1481] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:08:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 01:08:24.075664 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5651fa6-9704-4ebd-888e-0571b601fca4 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:08:24.076247 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5651fa6-9704-4ebd-888e-0571b601fca4 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:08:24.076373 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5651fa6-9704-4ebd-888e-0571b601fca4 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:08:24.077620 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 729/1482] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:08:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:08:24.082871 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e1027-4e66-456c-87fc-0309ce869ffa None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:08:24.083326 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e1027-4e66-456c-87fc-0309ce869ffa None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:08:24.083434 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e1027-4e66-456c-87fc-0309ce869ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:08:24.149628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-995e1027-4e66-456c-87fc-0309ce869ffa None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:08:24.151716 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-995e1027-4e66-456c-87fc-0309ce869ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:08:24.197121 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:08:24.248948 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 754/1483] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:08:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:08:24.535655 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5651fa6-9704-4ebd-888e-0571b601fca4 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:08:24.537238 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-b5651fa6-9704-4ebd-888e-0571b601fca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:08:24.561289 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:08:24.640816 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5651fa6-9704-4ebd-888e-0571b601fca4 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:08:24.716756 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06399b2c-b6fa-40a1-8f73-b8010279e03d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:08:24.717355 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06399b2c-b6fa-40a1-8f73-b8010279e03d service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:08:24.717632 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06399b2c-b6fa-40a1-8f73-b8010279e03d service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:08:24.717899 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06399b2c-b6fa-40a1-8f73-b8010279e03d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:08:24.719489 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-06399b2c-b6fa-40a1-8f73-b8010279e03d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:08:24.793485 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:08:24.827411 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06399b2c-b6fa-40a1-8f73-b8010279e03d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:08:24.828885 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06399b2c-b6fa-40a1-8f73-b8010279e03d service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:08:24.846774 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 730/1484] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:08:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:08:27.059955 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782002b-c377-4d0e-88fc-8e2ed420fff7 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:08:27.060531 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782002b-c377-4d0e-88fc-8e2ed420fff7 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:08:27.060531 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782002b-c377-4d0e-88fc-8e2ed420fff7 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:08:27.061888 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 755/1485] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:08:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:08:27.067148 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfddea0-c6c5-484d-82da-1bbd53be2f6f None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:08:27.067611 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfddea0-c6c5-484d-82da-1bbd53be2f6f None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:08:27.067718 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfddea0-c6c5-484d-82da-1bbd53be2f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:08:27.134331 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-8bfddea0-c6c5-484d-82da-1bbd53be2f6f None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:08:27.135560 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-8bfddea0-c6c5-484d-82da-1bbd53be2f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:08:27.232121 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 731/1486] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:08:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:08:27.525156 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8782002b-c377-4d0e-88fc-8e2ed420fff7 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:08:27.527301 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-8782002b-c377-4d0e-88fc-8e2ed420fff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:08:27.631125 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8782002b-c377-4d0e-88fc-8e2ed420fff7 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:08:27.703435 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:08:27.704947 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e6cd63-b8e3-4fbd-99ff-d29967c9ce6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:08:27.705782 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6cd63-b8e3-4fbd-99ff-d29967c9ce6e service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:08:27.706254 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6cd63-b8e3-4fbd-99ff-d29967c9ce6e service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:08:27.706692 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6cd63-b8e3-4fbd-99ff-d29967c9ce6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:08:27.708486 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-09e6cd63-b8e3-4fbd-99ff-d29967c9ce6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:08:27.817841 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e6cd63-b8e3-4fbd-99ff-d29967c9ce6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:08:27.819191 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e6cd63-b8e3-4fbd-99ff-d29967c9ce6e service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:08:27.837237 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 756/1487] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:08:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:08:30.147002 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4520f-f38f-465a-8b74-27e2198625f9 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:08:30.147575 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4520f-f38f-465a-8b74-27e2198625f9 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:08:30.147575 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4520f-f38f-465a-8b74-27e2198625f9 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:08:30.148974 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 732/1488] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:08:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:08:30.155222 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995846d-3c33-4cac-b67c-2a77c5b94980 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:08:30.155662 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995846d-3c33-4cac-b67c-2a77c5b94980 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:08:30.155771 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995846d-3c33-4cac-b67c-2a77c5b94980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:08:30.221436 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-e995846d-3c33-4cac-b67c-2a77c5b94980 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:08:30.222790 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-e995846d-3c33-4cac-b67c-2a77c5b94980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:08:30.277397 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:08:30.321168 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 757/1489] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:08:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:08:30.613934 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b4520f-f38f-465a-8b74-27e2198625f9 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:08:30.615477 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-a6b4520f-f38f-465a-8b74-27e2198625f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:08:30.653468 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:08:30.720498 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b4520f-f38f-465a-8b74-27e2198625f9 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:08:30.794733 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159e4c02-5952-4eb1-a8c3-ee5699026003 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:08:30.795327 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e4c02-5952-4eb1-a8c3-ee5699026003 service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:08:30.795602 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e4c02-5952-4eb1-a8c3-ee5699026003 service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:08:30.795872 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e4c02-5952-4eb1-a8c3-ee5699026003 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:08:30.797483 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-159e4c02-5952-4eb1-a8c3-ee5699026003 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:08:30.875601 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:08:30.904769 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159e4c02-5952-4eb1-a8c3-ee5699026003 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:08:30.906284 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159e4c02-5952-4eb1-a8c3-ee5699026003 service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:08:30.925498 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 733/1490] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:08:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:08:33.335231 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae2afd3-0a58-4960-b9ad-f8638dcc280b None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:08:33.335794 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae2afd3-0a58-4960-b9ad-f8638dcc280b None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:08:33.335794 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae2afd3-0a58-4960-b9ad-f8638dcc280b None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:08:33.337150 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 758/1491] 10.222.0.42 () {54 vars in 927 bytes} [Thu Sep 8 01:08:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:08:33.343291 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdaee74-8f92-40e3-9992-3d3c25814c4b None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:08:33.343702 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdaee74-8f92-40e3-9992-3d3c25814c4b None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:08:33.343810 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdaee74-8f92-40e3-9992-3d3c25814c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:08:33.409174 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-bcdaee74-8f92-40e3-9992-3d3c25814c4b None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:08:33.410383 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-bcdaee74-8f92-40e3-9992-3d3c25814c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:08:33.504528 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 734/1492] 10.222.0.42 () {58 vars in 1024 bytes} [Thu Sep 8 01:08:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:08:33.807012 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae2afd3-0a58-4960-b9ad-f8638dcc280b None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:08:33.808573 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-8ae2afd3-0a58-4960-b9ad-f8638dcc280b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:08:33.910848 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae2afd3-0a58-4960-b9ad-f8638dcc280b None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:08:33.984718 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17beb51a-4301-48f7-b51c-82dbbcf04c6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:08:33.985313 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17beb51a-4301-48f7-b51c-82dbbcf04c6b service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:08:33.985586 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17beb51a-4301-48f7-b51c-82dbbcf04c6b service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:08:33.985855 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17beb51a-4301-48f7-b51c-82dbbcf04c6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:08:33.987435 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-17beb51a-4301-48f7-b51c-82dbbcf04c6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:08:33.996552 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:08:34.099482 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17beb51a-4301-48f7-b51c-82dbbcf04c6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:08:34.100953 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17beb51a-4301-48f7-b51c-82dbbcf04c6b service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:08:34.120247 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 759/1493] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:08:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:08:43.503029 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=82 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:08:44.117296 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=50 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:09:48.878828 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ab4704-c338-453f-8246-568a9116c306 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:09:48.879409 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ab4704-c338-453f-8246-568a9116c306 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:09:48.879409 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ab4704-c338-453f-8246-568a9116c306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:09:48.954192 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-22ab4704-c338-453f-8246-568a9116c306 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:09:48.955406 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-22ab4704-c338-453f-8246-568a9116c306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:09:49.053065 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 735/1494] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:09:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:09:49.061940 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6456cf-f120-414b-ac35-837a5e30995b None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:09:49.062430 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6456cf-f120-414b-ac35-837a5e30995b None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:09:49.062558 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6456cf-f120-414b-ac35-837a5e30995b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:09:49.138095 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-4d6456cf-f120-414b-ac35-837a5e30995b None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:09:49.139298 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-4d6456cf-f120-414b-ac35-837a5e30995b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:09:49.237223 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 760/1495] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:09:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:09:49.242499 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ab4704-c338-453f-8246-568a9116c306 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:09:49.243802 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-22ab4704-c338-453f-8246-568a9116c306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:09:49.255367 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ab4704-c338-453f-8246-568a9116c306 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:09:49.317274 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:09:49.352369 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc54d9f-3f7c-4830-81bd-db0736d489b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:09:49.352967 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc54d9f-3f7c-4830-81bd-db0736d489b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:09:49.353243 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc54d9f-3f7c-4830-81bd-db0736d489b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:09:49.353509 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc54d9f-3f7c-4830-81bd-db0736d489b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:09:49.354149 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc54d9f-3f7c-4830-81bd-db0736d489b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:09:49.354447 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc54d9f-3f7c-4830-81bd-db0736d489b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:09:49.355458 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: WARNING py.warnings [None req-0fc54d9f-3f7c-4830-81bd-db0736d489b3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 08 01:09:49.355458 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: warnings.warn(msg) Sep 08 01:09:49.355458 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: Sep 08 01:09:49.356358 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc54d9f-3f7c-4830-81bd-db0736d489b3 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:09:49.367298 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 736/1496] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:09:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:09:49.375719 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6456cf-f120-414b-ac35-837a5e30995b None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:09:49.384943 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6456cf-f120-414b-ac35-837a5e30995b None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:09:49.445882 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:09:49.479779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d04fe04-1e5f-42a1-a8f2-e44f0b46ff97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:09:49.480398 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04fe04-1e5f-42a1-a8f2-e44f0b46ff97 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:09:49.480674 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04fe04-1e5f-42a1-a8f2-e44f0b46ff97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:09:49.480943 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04fe04-1e5f-42a1-a8f2-e44f0b46ff97 admin admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:09:49.481560 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d04fe04-1e5f-42a1-a8f2-e44f0b46ff97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:09:49.483296 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: WARNING py.warnings [None req-1d04fe04-1e5f-42a1-a8f2-e44f0b46ff97 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 01:09:49.483296 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: warnings.warn(msg) Sep 08 01:09:49.483296 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: Sep 08 01:09:49.484504 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d04fe04-1e5f-42a1-a8f2-e44f0b46ff97 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:09:49.540675 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 761/1497] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:09:49 2022] POST /identity/v3/projects => generated 377 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:09:49.545096 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc54d9f-3f7c-4830-81bd-db0736d489b3 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:09:49.546608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-0fc54d9f-3f7c-4830-81bd-db0736d489b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:09:49.580522 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc54d9f-3f7c-4830-81bd-db0736d489b3 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:09:49.674160 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2d2eee-dedb-4cf9-9e6a-53aa748265f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:09:49.674762 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2d2eee-dedb-4cf9-9e6a-53aa748265f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:09:49.675039 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2d2eee-dedb-4cf9-9e6a-53aa748265f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:09:49.675307 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2d2eee-dedb-4cf9-9e6a-53aa748265f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:09:49.675926 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2d2eee-dedb-4cf9-9e6a-53aa748265f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:09:49.676990 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: WARNING py.warnings [None req-ab2d2eee-dedb-4cf9-9e6a-53aa748265f6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 01:09:49.676990 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: warnings.warn(msg) Sep 08 01:09:49.676990 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: Sep 08 01:09:49.678192 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2d2eee-dedb-4cf9-9e6a-53aa748265f6 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:09:49.720409 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 737/1498] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:09:49 2022] POST /identity/v3/users => generated 289 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:09:49.724490 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d04fe04-1e5f-42a1-a8f2-e44f0b46ff97 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:09:49.726505 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-1d04fe04-1e5f-42a1-a8f2-e44f0b46ff97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:09:49.788112 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d04fe04-1e5f-42a1-a8f2-e44f0b46ff97 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:09:49.870563 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:09:49.882763 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eebf3b-0065-4a40-b576-f09348c36c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:09:49.883530 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eebf3b-0065-4a40-b576-f09348c36c78 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:09:49.883809 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eebf3b-0065-4a40-b576-f09348c36c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:09:49.884077 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eebf3b-0065-4a40-b576-f09348c36c78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:09:49.884738 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eebf3b-0065-4a40-b576-f09348c36c78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:09:49.886009 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: WARNING py.warnings [None req-39eebf3b-0065-4a40-b576-f09348c36c78 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 08 01:09:49.886009 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: warnings.warn(msg) Sep 08 01:09:49.886009 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: Sep 08 01:09:49.886887 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eebf3b-0065-4a40-b576-f09348c36c78 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:09:49.899627 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 762/1499] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:09:49 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:09:49.903608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2d2eee-dedb-4cf9-9e6a-53aa748265f6 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:09:49.905204 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-ab2d2eee-dedb-4cf9-9e6a-53aa748265f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:09:49.921543 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2d2eee-dedb-4cf9-9e6a-53aa748265f6 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:09:49.945963 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:09:50.016485 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbce997-ea33-4245-b7c7-c40650191c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:09:50.017082 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbce997-ea33-4245-b7c7-c40650191c81 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:09:50.017354 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbce997-ea33-4245-b7c7-c40650191c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:09:50.017619 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbce997-ea33-4245-b7c7-c40650191c81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:09:50.018246 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbce997-ea33-4245-b7c7-c40650191c81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:09:50.019243 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: WARNING py.warnings [None req-adbce997-ea33-4245-b7c7-c40650191c81 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 08 01:09:50.019243 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: warnings.warn(msg) Sep 08 01:09:50.019243 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: Sep 08 01:09:50.020119 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbce997-ea33-4245-b7c7-c40650191c81 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:09:50.032530 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 738/1500] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:09:49 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:09:50.036786 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eebf3b-0065-4a40-b576-f09348c36c78 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:09:50.038318 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-39eebf3b-0065-4a40-b576-f09348c36c78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:09:50.072649 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eebf3b-0065-4a40-b576-f09348c36c78 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:09:50.165992 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a96039-a00a-4289-86bd-2f6d28865322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:09:50.166576 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a96039-a00a-4289-86bd-2f6d28865322 admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:09:50.166852 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a96039-a00a-4289-86bd-2f6d28865322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:09:50.167128 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a96039-a00a-4289-86bd-2f6d28865322 admin admin] PATH_INFO: `/v3/projects/9f1c95b22c4a47e9a925053e03b5619d/users/216b1757d249478291984a8009c4335a/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:09:50.190270 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a96039-a00a-4289-86bd-2f6d28865322 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f1c95b22c4a47e9a925053e03b5619d, user_id=216b1757d249478291984a8009c4335a, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:09:50.191304 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: WARNING py.warnings [None req-98a96039-a00a-4289-86bd-2f6d28865322 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 01:09:50.191304 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: warnings.warn(msg) Sep 08 01:09:50.191304 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: Sep 08 01:09:50.193306 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a96039-a00a-4289-86bd-2f6d28865322 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:09:50.219406 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 763/1501] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:09:50 2022] PUT /identity/v3/projects/9f1c95b22c4a47e9a925053e03b5619d/users/216b1757d249478291984a8009c4335a/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:09:50.231829 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1bf944-95ca-4dee-b585-04715f5bc1df None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:09:50.232123 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1bf944-95ca-4dee-b585-04715f5bc1df None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:09:50.232447 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1bf944-95ca-4dee-b585-04715f5bc1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:09:50.291285 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-5e1bf944-95ca-4dee-b585-04715f5bc1df None None] MFA Rules not processed for user `216b1757d249478291984a8009c4335a`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:09:50.292551 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-5e1bf944-95ca-4dee-b585-04715f5bc1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:09:50.371291 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 739/1502] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:09:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:09:50.384771 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b86d400-e61e-4585-9ce1-bbb62ef9c690 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:09:50.385064 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b86d400-e61e-4585-9ce1-bbb62ef9c690 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:09:50.385599 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b86d400-e61e-4585-9ce1-bbb62ef9c690 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:09:50.387008 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 764/1503] 10.222.0.42 () {54 vars in 964 bytes} [Thu Sep 8 01:09:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:09:50.391615 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cb66f3-ddd6-42c9-9977-7def363cc8c3 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:09:50.391902 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cb66f3-ddd6-42c9-9977-7def363cc8c3 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:09:50.392192 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cb66f3-ddd6-42c9-9977-7def363cc8c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:09:50.423950 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:09:50.468404 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-21cb66f3-ddd6-42c9-9977-7def363cc8c3 None None] MFA Rules not processed for user `a307e9150ead4877b970e09f36553968`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:09:50.469624 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-21cb66f3-ddd6-42c9-9977-7def363cc8c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:09:50.544618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 740/1504] 10.222.0.42 () {58 vars in 1061 bytes} [Thu Sep 8 01:09:50 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:09:50.552449 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b86d400-e61e-4585-9ce1-bbb62ef9c690 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:09:50.555112 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-4b86d400-e61e-4585-9ce1-bbb62ef9c690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:09:50.604499 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:09:50.659262 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b86d400-e61e-4585-9ce1-bbb62ef9c690 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:09:50.733966 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1d3796-e202-46dc-a332-b7f630221cef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:09:50.734500 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1d3796-e202-46dc-a332-b7f630221cef service cinder] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:09:50.734777 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1d3796-e202-46dc-a332-b7f630221cef service cinder] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:09:50.735044 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1d3796-e202-46dc-a332-b7f630221cef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:09:50.736664 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-ae1d3796-e202-46dc-a332-b7f630221cef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:09:50.837938 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1d3796-e202-46dc-a332-b7f630221cef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:09:50.839378 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1d3796-e202-46dc-a332-b7f630221cef service cinder] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:09:50.842866 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:09:50.858109 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 765/1505] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:09:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:09:50.905157 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828ffd9-51b0-4a3e-87f5-aee3998b86b2 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:09:50.905578 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828ffd9-51b0-4a3e-87f5-aee3998b86b2 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:09:50.905712 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828ffd9-51b0-4a3e-87f5-aee3998b86b2 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:09:50.906968 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 741/1506] 10.222.0.42 () {52 vars in 830 bytes} [Thu Sep 8 01:09:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:09:50.910550 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1d3796-e202-46dc-a332-b7f630221cef service cinder] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:09:50.911784 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-ae1d3796-e202-46dc-a332-b7f630221cef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:09:51.010600 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1d3796-e202-46dc-a332-b7f630221cef service cinder] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:09:51.103751 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e6af00-603b-4e26-96ea-c633becc78f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:09:51.104442 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e6af00-603b-4e26-96ea-c633becc78f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:09:51.104667 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e6af00-603b-4e26-96ea-c633becc78f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:09:51.104941 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e6af00-603b-4e26-96ea-c633becc78f6 admin admin] PATH_INFO: `/v3/users/216b1757d249478291984a8009c4335a` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:09:51.118427 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e6af00-603b-4e26-96ea-c633becc78f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=216b1757d249478291984a8009c4335a)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:09:51.119453 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: WARNING py.warnings [None req-a9e6af00-603b-4e26-96ea-c633becc78f6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 01:09:51.119453 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: warnings.warn(msg) Sep 08 01:09:51.119453 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: Sep 08 01:09:51.120709 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e6af00-603b-4e26-96ea-c633becc78f6 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:09:51.130285 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:09:51.188766 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-a9e6af00-603b-4e26-96ea-c633becc78f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '216b1757d249478291984a8009c4335a'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:09:51.213409 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-a9e6af00-603b-4e26-96ea-c633becc78f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '216b1757d249478291984a8009c4335a'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:09:51.227401 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-a9e6af00-603b-4e26-96ea-c633becc78f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '216b1757d249478291984a8009c4335a'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:09:51.236600 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-a9e6af00-603b-4e26-96ea-c633becc78f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '216b1757d249478291984a8009c4335a'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:09:51.237032 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-a9e6af00-603b-4e26-96ea-c633becc78f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '216b1757d249478291984a8009c4335a'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:09:51.242933 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 766/1507] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:09:50 2022] DELETE /identity/v3/users/216b1757d249478291984a8009c4335a => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:09:51.247474 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1915fa5-c56e-4eb5-9743-89ec2e0a5880 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:09:51.247762 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1915fa5-c56e-4eb5-9743-89ec2e0a5880 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:09:51.248040 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1915fa5-c56e-4eb5-9743-89ec2e0a5880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:09:51.305051 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-b1915fa5-c56e-4eb5-9743-89ec2e0a5880 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:09:51.306191 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-b1915fa5-c56e-4eb5-9743-89ec2e0a5880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:09:51.382580 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:09:51.396503 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 742/1508] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:09:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:09:51.406899 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e6af00-603b-4e26-96ea-c633becc78f6 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:09:51.408114 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-a9e6af00-603b-4e26-96ea-c633becc78f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:09:51.510407 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e6af00-603b-4e26-96ea-c633becc78f6 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:09:51.564480 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:09:51.586053 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf75eb1e-813e-449e-8f2d-01413bd07046 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:09:51.586685 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75eb1e-813e-449e-8f2d-01413bd07046 service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:09:51.586959 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75eb1e-813e-449e-8f2d-01413bd07046 service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:09:51.587226 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75eb1e-813e-449e-8f2d-01413bd07046 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:09:51.588865 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-bf75eb1e-813e-449e-8f2d-01413bd07046 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:09:51.697902 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf75eb1e-813e-449e-8f2d-01413bd07046 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:09:51.699352 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf75eb1e-813e-449e-8f2d-01413bd07046 service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:09:51.718333 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 767/1509] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:09:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:09:52.084749 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1915fa5-c56e-4eb5-9743-89ec2e0a5880 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:09:52.086312 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-b1915fa5-c56e-4eb5-9743-89ec2e0a5880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:09:52.099924 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1915fa5-c56e-4eb5-9743-89ec2e0a5880 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:09:52.196892 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f41cbda-2c95-4ff1-a569-22f8286474d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:09:52.197514 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f41cbda-2c95-4ff1-a569-22f8286474d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:09:52.197788 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f41cbda-2c95-4ff1-a569-22f8286474d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:09:52.198054 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f41cbda-2c95-4ff1-a569-22f8286474d9 admin admin] PATH_INFO: `/v3/projects/9f1c95b22c4a47e9a925053e03b5619d` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:09:52.208254 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f41cbda-2c95-4ff1-a569-22f8286474d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f1c95b22c4a47e9a925053e03b5619d)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:09:52.209280 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: WARNING py.warnings [None req-0f41cbda-2c95-4ff1-a569-22f8286474d9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 01:09:52.209280 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: warnings.warn(msg) Sep 08 01:09:52.209280 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: Sep 08 01:09:52.211144 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f41cbda-2c95-4ff1-a569-22f8286474d9 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:09:52.269889 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-0f41cbda-2c95-4ff1-a569-22f8286474d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f1c95b22c4a47e9a925053e03b5619d', 'request_id': 'req-0f41cbda-2c95-4ff1-a569-22f8286474d9'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:09:52.281163 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-0f41cbda-2c95-4ff1-a569-22f8286474d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f1c95b22c4a47e9a925053e03b5619d', 'request_id': 'req-0f41cbda-2c95-4ff1-a569-22f8286474d9'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:09:52.295033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-0f41cbda-2c95-4ff1-a569-22f8286474d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f1c95b22c4a47e9a925053e03b5619d', 'request_id': 'req-0f41cbda-2c95-4ff1-a569-22f8286474d9'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:09:52.316992 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-0f41cbda-2c95-4ff1-a569-22f8286474d9 admin admin] The token cache is being invalidate because project 9f1c95b22c4a47e9a925053e03b5619d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81661) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:09:52.317277 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-0f41cbda-2c95-4ff1-a569-22f8286474d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:09:52.317704 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-0f41cbda-2c95-4ff1-a569-22f8286474d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:09:52.319090 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 743/1510] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:09:52 2022] DELETE /identity/v3/projects/9f1c95b22c4a47e9a925053e03b5619d => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:10:01.716062 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=62 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:10:02.315043 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:11:20.097243 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f843c56-021b-4c20-b5df-defb7631dd27 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:11:20.097856 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f843c56-021b-4c20-b5df-defb7631dd27 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:11:20.097856 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f843c56-021b-4c20-b5df-defb7631dd27 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:11:20.099641 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 768/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 01:11:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 01:11:20.104578 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0391ad-988c-4570-a671-744ab40186bc None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:11:20.104988 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0391ad-988c-4570-a671-744ab40186bc None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:11:20.105122 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0391ad-988c-4570-a671-744ab40186bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:11:20.184447 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-2d0391ad-988c-4570-a671-744ab40186bc None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:11:20.185656 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-2d0391ad-988c-4570-a671-744ab40186bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:11:20.294219 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 744/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 01:11:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:11:20.448884 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f843c56-021b-4c20-b5df-defb7631dd27 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:11:20.451090 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-7f843c56-021b-4c20-b5df-defb7631dd27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:11:20.569626 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f843c56-021b-4c20-b5df-defb7631dd27 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:11:20.575430 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3203009e-4611-4239-949c-c5ac6ef350c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae643cfdcb164aa9b474747cd21aa116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:11:20.576035 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3203009e-4611-4239-949c-c5ac6ef350c3 service nova] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:11:20.576346 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3203009e-4611-4239-949c-c5ac6ef350c3 service nova] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:11:20.576623 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3203009e-4611-4239-949c-c5ac6ef350c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:11:20.585343 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3203009e-4611-4239-949c-c5ac6ef350c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:11:20.586929 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3203009e-4611-4239-949c-c5ac6ef350c3 service nova] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:11:20.596284 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 769/1513] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:11:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:11:21.573719 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0dbab-e75e-4ed3-807c-1ac85c2ca72a None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:11:21.574286 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0dbab-e75e-4ed3-807c-1ac85c2ca72a None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:11:21.574286 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0dbab-e75e-4ed3-807c-1ac85c2ca72a None None] PATH_INFO: `/v3` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:11:21.575542 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 745/1514] 10.222.0.46 () {54 vars in 925 bytes} [Thu Sep 8 01:11:21 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:11:21.585540 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c6949-8b0b-4250-86de-0b3bdb0e36e3 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:11:21.586065 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c6949-8b0b-4250-86de-0b3bdb0e36e3 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:11:21.586065 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c6949-8b0b-4250-86de-0b3bdb0e36e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:11:21.654081 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-f48c6949-8b0b-4250-86de-0b3bdb0e36e3 None None] MFA Rules not processed for user `301d30bbd3de4cc69e96bb12729442c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:11:21.655294 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-f48c6949-8b0b-4250-86de-0b3bdb0e36e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:11:21.685592 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c4360-ac4b-4d3c-bd9c-d1d09fad549f None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:11:21.685871 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c4360-ac4b-4d3c-bd9c-d1d09fad549f None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:11:21.686144 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c4360-ac4b-4d3c-bd9c-d1d09fad549f None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:11:21.687506 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 746/1515] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 01:11:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 01:11:21.694271 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69172a4a-a7ed-4c38-b936-4096c3c379b7 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:11:21.694551 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69172a4a-a7ed-4c38-b936-4096c3c379b7 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:11:21.694829 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69172a4a-a7ed-4c38-b936-4096c3c379b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:11:21.715496 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-69172a4a-a7ed-4c38-b936-4096c3c379b7 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:11:21.716736 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-69172a4a-a7ed-4c38-b936-4096c3c379b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:11:21.724997 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 747/1516] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 01:11:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:11:21.725177 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:11:21.731385 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 770/1517] 10.222.0.46 () {58 vars in 1015 bytes} [Thu Sep 8 01:11:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:11:21.877581 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69172a4a-a7ed-4c38-b936-4096c3c379b7 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:11:21.890637 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69172a4a-a7ed-4c38-b936-4096c3c379b7 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:11:21.896908 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd84638-168b-4298-bfa5-92fe5f03bc64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae643cfdcb164aa9b474747cd21aa116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:11:21.897508 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd84638-168b-4298-bfa5-92fe5f03bc64 service nova] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:11:21.897784 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd84638-168b-4298-bfa5-92fe5f03bc64 service nova] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:11:21.898054 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd84638-168b-4298-bfa5-92fe5f03bc64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:11:21.906113 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd84638-168b-4298-bfa5-92fe5f03bc64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:11:21.907700 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd84638-168b-4298-bfa5-92fe5f03bc64 service nova] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:11:21.917068 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 748/1518] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:11:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:11:30.688062 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d59ed-c390-4a3b-acdc-91cecdc8c9b9 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:11:30.688572 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d59ed-c390-4a3b-acdc-91cecdc8c9b9 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:11:30.688676 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d59ed-c390-4a3b-acdc-91cecdc8c9b9 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:11:30.689971 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 771/1519] 10.222.0.42 () {54 vars in 961 bytes} [Thu Sep 8 01:11:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:11:30.695341 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4bab7-e935-46c6-8c34-6585addfeeb5 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:11:30.695785 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4bab7-e935-46c6-8c34-6585addfeeb5 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:11:30.695885 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4bab7-e935-46c6-8c34-6585addfeeb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:11:30.715184 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-43e4bab7-e935-46c6-8c34-6585addfeeb5 None None] MFA Rules not processed for user `301d30bbd3de4cc69e96bb12729442c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:11:30.716408 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-43e4bab7-e935-46c6-8c34-6585addfeeb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:11:30.725595 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 749/1520] 10.222.0.42 () {58 vars in 1058 bytes} [Thu Sep 8 01:11:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:11:30.733897 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d59ed-c390-4a3b-acdc-91cecdc8c9b9 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:11:30.747981 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d59ed-c390-4a3b-acdc-91cecdc8c9b9 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:11:30.753135 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b7a50-2e48-48be-90fb-a5c2504fddf1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '301d30bbd3de4cc69e96bb12729442c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:11:30.753726 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b7a50-2e48-48be-90fb-a5c2504fddf1 service placement] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:11:30.754006 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b7a50-2e48-48be-90fb-a5c2504fddf1 service placement] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:11:30.754278 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b7a50-2e48-48be-90fb-a5c2504fddf1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:11:30.762949 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0b7a50-2e48-48be-90fb-a5c2504fddf1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:11:30.764577 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0b7a50-2e48-48be-90fb-a5c2504fddf1 service placement] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:11:30.774308 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 772/1521] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:11:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 01:11:33.406148 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c49694-c405-425d-8456-cd3059fd05d2 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:11:33.406675 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c49694-c405-425d-8456-cd3059fd05d2 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:11:33.406675 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c49694-c405-425d-8456-cd3059fd05d2 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:11:33.407949 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 750/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 01:11:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:11:33.414064 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25241bf2-de90-4ad8-a92b-dd8a15e2c080 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:11:33.414491 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25241bf2-de90-4ad8-a92b-dd8a15e2c080 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:11:33.414597 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25241bf2-de90-4ad8-a92b-dd8a15e2c080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:11:33.435232 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-25241bf2-de90-4ad8-a92b-dd8a15e2c080 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:11:33.436477 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-25241bf2-de90-4ad8-a92b-dd8a15e2c080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:11:33.445134 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 773/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 01:11:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:11:33.598748 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c49694-c405-425d-8456-cd3059fd05d2 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:11:33.613421 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c49694-c405-425d-8456-cd3059fd05d2 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:11:33.619729 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d823a1a2-59e5-4c07-b061-6d9b303f3b10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae643cfdcb164aa9b474747cd21aa116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:11:33.620368 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823a1a2-59e5-4c07-b061-6d9b303f3b10 service nova] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:11:33.620659 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823a1a2-59e5-4c07-b061-6d9b303f3b10 service nova] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:11:33.620927 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823a1a2-59e5-4c07-b061-6d9b303f3b10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:11:33.629227 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d823a1a2-59e5-4c07-b061-6d9b303f3b10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:11:33.630839 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d823a1a2-59e5-4c07-b061-6d9b303f3b10 service nova] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:11:33.640709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 751/1524] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:11:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:11:34.720474 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b2b4f-bc46-4c19-9296-6e1bc6cb6e58 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:11:34.721029 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b2b4f-bc46-4c19-9296-6e1bc6cb6e58 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:11:34.721029 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b2b4f-bc46-4c19-9296-6e1bc6cb6e58 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:11:34.722408 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 774/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 01:11:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:11:34.728780 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68422e-32c3-4732-800e-12f4dc31e83d None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:11:34.729242 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68422e-32c3-4732-800e-12f4dc31e83d None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:11:34.729343 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68422e-32c3-4732-800e-12f4dc31e83d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:11:34.749016 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-0b68422e-32c3-4732-800e-12f4dc31e83d None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:11:34.750224 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-0b68422e-32c3-4732-800e-12f4dc31e83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:11:34.758504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 752/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 01:11:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:11:34.910465 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1b2b4f-bc46-4c19-9296-6e1bc6cb6e58 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:11:34.925008 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1b2b4f-bc46-4c19-9296-6e1bc6cb6e58 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:11:34.931241 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9001cb44-73de-4102-bec1-22f4b172efb9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae643cfdcb164aa9b474747cd21aa116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:11:34.931830 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9001cb44-73de-4102-bec1-22f4b172efb9 service nova] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:11:34.932105 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9001cb44-73de-4102-bec1-22f4b172efb9 service nova] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:11:34.932403 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9001cb44-73de-4102-bec1-22f4b172efb9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:11:34.940419 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9001cb44-73de-4102-bec1-22f4b172efb9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:11:34.942567 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9001cb44-73de-4102-bec1-22f4b172efb9 service nova] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:11:34.951765 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 775/1527] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:11:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:11:38.849288 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06148b4-41c3-4be9-a2e9-08c76e78b8df None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:11:38.849877 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06148b4-41c3-4be9-a2e9-08c76e78b8df None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:11:38.849877 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06148b4-41c3-4be9-a2e9-08c76e78b8df None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:11:38.851178 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 753/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 01:11:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:11:38.856439 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea063d68-c358-4bf9-9a20-761da20672cb None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:11:38.856851 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea063d68-c358-4bf9-9a20-761da20672cb None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:11:38.856986 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea063d68-c358-4bf9-9a20-761da20672cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:11:38.876233 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-ea063d68-c358-4bf9-9a20-761da20672cb None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:11:38.877423 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-ea063d68-c358-4bf9-9a20-761da20672cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:11:38.887265 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 776/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 01:11:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:11:39.178243 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06148b4-41c3-4be9-a2e9-08c76e78b8df None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:11:39.180054 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-d06148b4-41c3-4be9-a2e9-08c76e78b8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:11:39.208423 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:11:39.263655 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06148b4-41c3-4be9-a2e9-08c76e78b8df None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:11:39.267175 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fdc9fa-84f7-435b-bc05-4cefecf92e71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:11:39.267775 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fdc9fa-84f7-435b-bc05-4cefecf92e71 service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:11:39.268050 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fdc9fa-84f7-435b-bc05-4cefecf92e71 service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:11:39.268435 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fdc9fa-84f7-435b-bc05-4cefecf92e71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:11:39.277376 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fdc9fa-84f7-435b-bc05-4cefecf92e71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:11:39.278841 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fdc9fa-84f7-435b-bc05-4cefecf92e71 service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:11:39.288045 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 754/1530] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:11:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:11:41.442290 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b87c3-e421-4b87-92fc-66008991a251 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:11:41.442851 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b87c3-e421-4b87-92fc-66008991a251 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:11:41.442851 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b87c3-e421-4b87-92fc-66008991a251 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:11:41.444202 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 777/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 01:11:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:11:41.449448 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3b54f-5427-400b-95c4-28754d2f3eb9 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:11:41.449893 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3b54f-5427-400b-95c4-28754d2f3eb9 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:11:41.450004 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3b54f-5427-400b-95c4-28754d2f3eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:11:41.469882 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-03f3b54f-5427-400b-95c4-28754d2f3eb9 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:11:41.471085 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-03f3b54f-5427-400b-95c4-28754d2f3eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:11:41.478961 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 755/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 01:11:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:11:41.773720 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b87c3-e421-4b87-92fc-66008991a251 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:11:41.784522 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b87c3-e421-4b87-92fc-66008991a251 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:11:41.788818 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b5104-faea-42d0-a5d2-666756ca7fee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:11:41.789406 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b5104-faea-42d0-a5d2-666756ca7fee service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:11:41.789689 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b5104-faea-42d0-a5d2-666756ca7fee service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:11:41.789949 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b5104-faea-42d0-a5d2-666756ca7fee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:11:41.798729 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529b5104-faea-42d0-a5d2-666756ca7fee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:11:41.800162 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529b5104-faea-42d0-a5d2-666756ca7fee service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:11:41.810335 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 778/1533] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:11:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:11:45.445628 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898cff9e-4255-4f7f-8728-61827e6d5e97 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:11:45.446038 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898cff9e-4255-4f7f-8728-61827e6d5e97 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:11:45.446176 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898cff9e-4255-4f7f-8728-61827e6d5e97 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:11:45.447609 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 756/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 01:11:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:11:45.453082 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7266e4d-42d6-4e7d-bde3-4cdce65dd5f5 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:11:45.453525 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7266e4d-42d6-4e7d-bde3-4cdce65dd5f5 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:11:45.453628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7266e4d-42d6-4e7d-bde3-4cdce65dd5f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:11:45.475743 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-b7266e4d-42d6-4e7d-bde3-4cdce65dd5f5 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:11:45.476979 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-b7266e4d-42d6-4e7d-bde3-4cdce65dd5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:11:45.484841 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 779/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 01:11:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:11:45.792429 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898cff9e-4255-4f7f-8728-61827e6d5e97 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:11:45.794168 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-898cff9e-4255-4f7f-8728-61827e6d5e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:11:45.872530 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898cff9e-4255-4f7f-8728-61827e6d5e97 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:11:45.876085 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e9cf5b-ff69-4e8c-b769-c73c57b65569 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:11:45.876706 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e9cf5b-ff69-4e8c-b769-c73c57b65569 service cinder] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:11:45.876986 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e9cf5b-ff69-4e8c-b769-c73c57b65569 service cinder] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:11:45.877260 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e9cf5b-ff69-4e8c-b769-c73c57b65569 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:11:45.886540 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e9cf5b-ff69-4e8c-b769-c73c57b65569 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:11:45.887941 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e9cf5b-ff69-4e8c-b769-c73c57b65569 service cinder] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:11:45.898061 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 757/1536] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:11:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:11:48.044940 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aadb6d-c2f1-433a-b6d5-80938388ba53 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:11:48.045499 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aadb6d-c2f1-433a-b6d5-80938388ba53 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:11:48.045499 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aadb6d-c2f1-433a-b6d5-80938388ba53 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:11:48.046823 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 780/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 01:11:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:11:48.052919 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da0009b-1fb2-4f2c-9232-73f5d846e293 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:11:48.053341 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da0009b-1fb2-4f2c-9232-73f5d846e293 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:11:48.053467 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da0009b-1fb2-4f2c-9232-73f5d846e293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:11:48.072908 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-0da0009b-1fb2-4f2c-9232-73f5d846e293 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:11:48.074105 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-0da0009b-1fb2-4f2c-9232-73f5d846e293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:11:48.082482 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 758/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 01:11:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:11:48.384233 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aadb6d-c2f1-433a-b6d5-80938388ba53 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:11:48.386193 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-f9aadb6d-c2f1-433a-b6d5-80938388ba53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:11:48.399530 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aadb6d-c2f1-433a-b6d5-80938388ba53 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:11:48.403394 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ceef2a-bc47-48cc-b41b-71fc00e482d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:11:48.403992 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ceef2a-bc47-48cc-b41b-71fc00e482d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:11:48.404303 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ceef2a-bc47-48cc-b41b-71fc00e482d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:11:48.404574 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ceef2a-bc47-48cc-b41b-71fc00e482d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:11:48.413932 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ceef2a-bc47-48cc-b41b-71fc00e482d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:11:48.415358 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ceef2a-bc47-48cc-b41b-71fc00e482d8 service cinder] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:11:48.429933 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 781/1539] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:11:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:11:58.067992 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=66 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:11:58.424672 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=34 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:13.399173 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6019ee8c-3e74-4c60-ba2b-fba83685058b None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:13.399799 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6019ee8c-3e74-4c60-ba2b-fba83685058b None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:13.399799 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6019ee8c-3e74-4c60-ba2b-fba83685058b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:13.416226 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edad546-8bc8-4ad8-9531-4fb9b7c33a59 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:13.416836 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edad546-8bc8-4ad8-9531-4fb9b7c33a59 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:13.416836 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edad546-8bc8-4ad8-9531-4fb9b7c33a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:13.427041 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-6019ee8c-3e74-4c60-ba2b-fba83685058b None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:13.429088 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-6019ee8c-3e74-4c60-ba2b-fba83685058b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:13.444680 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 759/1540] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:13.452047 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-3edad546-8bc8-4ad8-9531-4fb9b7c33a59 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:13.452495 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85db4f-9352-4f29-a915-53c56795f55b None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:13.452783 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85db4f-9352-4f29-a915-53c56795f55b None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:13.453064 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85db4f-9352-4f29-a915-53c56795f55b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:13.454622 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-3edad546-8bc8-4ad8-9531-4fb9b7c33a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:13.466908 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 782/1541] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:13:13.468525 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f8678e-49c4-4be9-a01f-06ab4a43a1f7 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:13.469044 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f8678e-49c4-4be9-a01f-06ab4a43a1f7 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:13.469549 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f8678e-49c4-4be9-a01f-06ab4a43a1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:13.491167 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-5d85db4f-9352-4f29-a915-53c56795f55b None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:13.492440 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-5d85db4f-9352-4f29-a915-53c56795f55b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:13.503489 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 760/1542] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 08 01:13:13.505415 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d12d8-743c-4833-af45-6f6468b320b7 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:13.505921 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d12d8-743c-4833-af45-6f6468b320b7 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:13.506411 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d12d8-743c-4833-af45-6f6468b320b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:13.511612 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-61f8678e-49c4-4be9-a01f-06ab4a43a1f7 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:13.512869 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-61f8678e-49c4-4be9-a01f-06ab4a43a1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:13.530206 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-f95d12d8-743c-4833-af45-6f6468b320b7 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:13.532300 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-f95d12d8-743c-4833-af45-6f6468b320b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:13.542264 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 783/1543] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:13.542791 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 761/1544] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:13.543714 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1ea94-5e6c-4a28-a5a2-0ab7e88d7c95 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:13.544005 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1ea94-5e6c-4a28-a5a2-0ab7e88d7c95 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:13.544321 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1ea94-5e6c-4a28-a5a2-0ab7e88d7c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:13.544952 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e69b3c-356c-4e76-846b-ca3de2807592 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:13.545563 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e69b3c-356c-4e76-846b-ca3de2807592 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:13.546050 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e69b3c-356c-4e76-846b-ca3de2807592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:13.573867 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-01e69b3c-356c-4e76-846b-ca3de2807592 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:13.575508 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-01e69b3c-356c-4e76-846b-ca3de2807592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:13.585429 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-2dd1ea94-5e6c-4a28-a5a2-0ab7e88d7c95 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:13.586229 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 762/1545] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:13.587263 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e69b3c-356c-4e76-846b-ca3de2807592 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:13.592418 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-2dd1ea94-5e6c-4a28-a5a2-0ab7e88d7c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:13.599428 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e69b3c-356c-4e76-846b-ca3de2807592 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:13.602445 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 784/1546] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:13.603492 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd1ea94-5e6c-4a28-a5a2-0ab7e88d7c95 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:13.605292 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba3f96d-e9dc-4f9d-b74f-4b78a67ffa5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:13.606107 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba3f96d-e9dc-4f9d-b74f-4b78a67ffa5a admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:13.606692 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba3f96d-e9dc-4f9d-b74f-4b78a67ffa5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:13.607163 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba3f96d-e9dc-4f9d-b74f-4b78a67ffa5a admin admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:13.607999 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba3f96d-e9dc-4f9d-b74f-4b78a67ffa5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:13:13.608517 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba3f96d-e9dc-4f9d-b74f-4b78a67ffa5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:13.611195 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba3f96d-e9dc-4f9d-b74f-4b78a67ffa5a admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:13.614403 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd1ea94-5e6c-4a28-a5a2-0ab7e88d7c95 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:13.619680 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a09163-92b4-4036-8c50-b74a0e9a7726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:13.620425 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a09163-92b4-4036-8c50-b74a0e9a7726 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:13.620774 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a09163-92b4-4036-8c50-b74a0e9a7726 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:13.621142 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a09163-92b4-4036-8c50-b74a0e9a7726 admin admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:13.621870 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a09163-92b4-4036-8c50-b74a0e9a7726 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:13:13.622363 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a09163-92b4-4036-8c50-b74a0e9a7726 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:13.623457 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: WARNING py.warnings [None req-78a09163-92b4-4036-8c50-b74a0e9a7726 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 08 01:13:13.623457 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: warnings.warn(msg) Sep 08 01:13:13.623457 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: Sep 08 01:13:13.624509 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a09163-92b4-4036-8c50-b74a0e9a7726 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:13.629734 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 763/1547] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:13:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:13:13.631229 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a2f88-3592-41cb-9604-14148d3ad5a0 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:13.634511 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a2f88-3592-41cb-9604-14148d3ad5a0 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:13.634992 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a2f88-3592-41cb-9604-14148d3ad5a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:13.647875 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 785/1548] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:13:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:13:13.656193 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83481e4a-5d82-45d0-ab9c-80092ec7131a None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:13.656193 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83481e4a-5d82-45d0-ab9c-80092ec7131a None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:13.656193 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83481e4a-5d82-45d0-ab9c-80092ec7131a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:13.665985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-b65a2f88-3592-41cb-9604-14148d3ad5a0 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:13.667213 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-b65a2f88-3592-41cb-9604-14148d3ad5a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:13.677171 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 764/1549] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:13.677852 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-83481e4a-5d82-45d0-ab9c-80092ec7131a None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:13.678402 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65a2f88-3592-41cb-9604-14148d3ad5a0 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:13.679502 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-83481e4a-5d82-45d0-ab9c-80092ec7131a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:13.694533 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 786/1550] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:13.695314 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83481e4a-5d82-45d0-ab9c-80092ec7131a None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:13.697232 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65a2f88-3592-41cb-9604-14148d3ad5a0 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:13.711723 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83481e4a-5d82-45d0-ab9c-80092ec7131a None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:13.713228 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb36dc6-0930-487d-8c54-b9c106bbdc0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:13.714291 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb36dc6-0930-487d-8c54-b9c106bbdc0d admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:13.714840 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb36dc6-0930-487d-8c54-b9c106bbdc0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:13.715330 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb36dc6-0930-487d-8c54-b9c106bbdc0d admin admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:13.716319 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb36dc6-0930-487d-8c54-b9c106bbdc0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:13:13.716691 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaaf1c6-309a-4929-9144-b3289a8947f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:13.717089 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb36dc6-0930-487d-8c54-b9c106bbdc0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:13.717293 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaaf1c6-309a-4929-9144-b3289a8947f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:13.718076 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaaf1c6-309a-4929-9144-b3289a8947f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:13.718363 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaaf1c6-309a-4929-9144-b3289a8947f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:13.719456 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb36dc6-0930-487d-8c54-b9c106bbdc0d admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:13.719644 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaaf1c6-309a-4929-9144-b3289a8947f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:13.722499 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaaf1c6-309a-4929-9144-b3289a8947f4 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:13.737577 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 765/1551] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:13:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:13:13.738480 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb36dc6-0930-487d-8c54-b9c106bbdc0d admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:13.750037 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb36dc6-0930-487d-8c54-b9c106bbdc0d admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:13.755413 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e73a41-3694-4d28-a986-a124296829bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:13.755996 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e73a41-3694-4d28-a986-a124296829bd admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:13.756303 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e73a41-3694-4d28-a986-a124296829bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:13.756606 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e73a41-3694-4d28-a986-a124296829bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:13.757214 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e73a41-3694-4d28-a986-a124296829bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:13.758248 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: WARNING py.warnings [None req-71e73a41-3694-4d28-a986-a124296829bd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 01:13:13.758248 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: warnings.warn(msg) Sep 08 01:13:13.758248 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: Sep 08 01:13:13.759925 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e73a41-3694-4d28-a986-a124296829bd admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:13.768926 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 787/1552] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:13:13 2022] POST /identity/v3/projects => generated 387 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:13:13.768926 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2898e-3a5b-43bc-ad74-2b76ea87bcc7 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:13.768926 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2898e-3a5b-43bc-ad74-2b76ea87bcc7 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:13.768926 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2898e-3a5b-43bc-ad74-2b76ea87bcc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:13.789672 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-98a2898e-3a5b-43bc-ad74-2b76ea87bcc7 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:13.789908 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 766/1553] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:13:13 2022] POST /identity/v3/projects => generated 389 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:13:13.790974 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-98a2898e-3a5b-43bc-ad74-2b76ea87bcc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:13.791265 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e73a41-3694-4d28-a986-a124296829bd admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:13.852814 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 788/1554] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:13.854059 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da6fd7b-63f0-4944-96fe-e3612e58b45a None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:13.854337 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da6fd7b-63f0-4944-96fe-e3612e58b45a None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:13.854605 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da6fd7b-63f0-4944-96fe-e3612e58b45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:13.861348 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e73a41-3694-4d28-a986-a124296829bd admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:13.866287 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f839d6-3ea1-4024-8ac5-b620716f7933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:13.867004 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f839d6-3ea1-4024-8ac5-b620716f7933 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:13.867289 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f839d6-3ea1-4024-8ac5-b620716f7933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:13.867568 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f839d6-3ea1-4024-8ac5-b620716f7933 admin admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:13.868247 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f839d6-3ea1-4024-8ac5-b620716f7933 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:13:13.868552 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f839d6-3ea1-4024-8ac5-b620716f7933 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:13.870243 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f839d6-3ea1-4024-8ac5-b620716f7933 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:13.873973 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-6da6fd7b-63f0-4944-96fe-e3612e58b45a None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:13.875193 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-6da6fd7b-63f0-4944-96fe-e3612e58b45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:13.888941 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 767/1555] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:13:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:13:13.888941 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7c8f3-ebba-40eb-b1eb-5ac748ae909c None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:13.888941 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7c8f3-ebba-40eb-b1eb-5ac748ae909c None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:13.888941 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7c8f3-ebba-40eb-b1eb-5ac748ae909c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:13.890508 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 789/1556] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:13.891302 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da6fd7b-63f0-4944-96fe-e3612e58b45a None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:13.902901 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da6fd7b-63f0-4944-96fe-e3612e58b45a None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:13.907642 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9d6c06-77c7-4862-87a9-22d8cd0a5b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:13.908814 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d6c06-77c7-4862-87a9-22d8cd0a5b3e admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:13.909088 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d6c06-77c7-4862-87a9-22d8cd0a5b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:13.909387 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d6c06-77c7-4862-87a9-22d8cd0a5b3e admin admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:13.909530 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-a4c7c8f3-ebba-40eb-b1eb-5ac748ae909c None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:13.910018 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9d6c06-77c7-4862-87a9-22d8cd0a5b3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:13.910747 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-a4c7c8f3-ebba-40eb-b1eb-5ac748ae909c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:13.911969 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9d6c06-77c7-4862-87a9-22d8cd0a5b3e admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:13.918969 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 768/1557] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:13.920351 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9740798a-92f4-4bf1-baf3-4b0293c14d99 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:13.920684 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9740798a-92f4-4bf1-baf3-4b0293c14d99 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:13.921013 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9740798a-92f4-4bf1-baf3-4b0293c14d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:13.943099 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-9740798a-92f4-4bf1-baf3-4b0293c14d99 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:13.947404 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 790/1558] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:13:13 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:13:13.947560 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-9740798a-92f4-4bf1-baf3-4b0293c14d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:13.947670 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9d6c06-77c7-4862-87a9-22d8cd0a5b3e admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:14.005980 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 769/1559] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:14.006681 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9740798a-92f4-4bf1-baf3-4b0293c14d99 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:14.008604 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9d6c06-77c7-4862-87a9-22d8cd0a5b3e admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:14.014090 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f48fb3-dd48-4956-8d5f-a1007a1d97c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:14.014871 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f48fb3-dd48-4956-8d5f-a1007a1d97c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:14.015158 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f48fb3-dd48-4956-8d5f-a1007a1d97c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:14.015431 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f48fb3-dd48-4956-8d5f-a1007a1d97c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:14.016036 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9740798a-92f4-4bf1-baf3-4b0293c14d99 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:14.016173 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f48fb3-dd48-4956-8d5f-a1007a1d97c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:14.017112 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: WARNING py.warnings [None req-56f48fb3-dd48-4956-8d5f-a1007a1d97c0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 01:13:14.017112 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: warnings.warn(msg) Sep 08 01:13:14.017112 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: Sep 08 01:13:14.018909 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f48fb3-dd48-4956-8d5f-a1007a1d97c0 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:14.021442 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7306ca-3701-4400-86f2-57026d4939fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:14.022039 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7306ca-3701-4400-86f2-57026d4939fe admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:14.022341 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7306ca-3701-4400-86f2-57026d4939fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:14.022621 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7306ca-3701-4400-86f2-57026d4939fe admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:14.023245 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7306ca-3701-4400-86f2-57026d4939fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:14.025204 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7306ca-3701-4400-86f2-57026d4939fe admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:14.028156 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:14.033702 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:14.040691 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 791/1560] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:13:13 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:13:14.044475 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f48fb3-dd48-4956-8d5f-a1007a1d97c0 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:14.048382 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 770/1561] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:13:14 2022] POST /identity/v3/users => generated 295 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:13:14.049510 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7306ca-3701-4400-86f2-57026d4939fe admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:14.057625 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f48fb3-dd48-4956-8d5f-a1007a1d97c0 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:14.058891 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7306ca-3701-4400-86f2-57026d4939fe admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:14.062354 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d44ec8b-8551-45ff-8f5c-f4e670fe69d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:14.062984 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d44ec8b-8551-45ff-8f5c-f4e670fe69d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:14.063266 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d44ec8b-8551-45ff-8f5c-f4e670fe69d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:14.063514 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e79d96-66a8-464c-9b5a-182ec440321c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:14.063647 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d44ec8b-8551-45ff-8f5c-f4e670fe69d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:14.064650 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e79d96-66a8-464c-9b5a-182ec440321c admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:14.064844 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d44ec8b-8551-45ff-8f5c-f4e670fe69d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:13:14.064993 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e79d96-66a8-464c-9b5a-182ec440321c admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:14.065161 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d44ec8b-8551-45ff-8f5c-f4e670fe69d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:14.065358 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e79d96-66a8-464c-9b5a-182ec440321c admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:14.065798 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e79d96-66a8-464c-9b5a-182ec440321c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:14.066672 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d44ec8b-8551-45ff-8f5c-f4e670fe69d8 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:14.067709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e79d96-66a8-464c-9b5a-182ec440321c admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:14.079045 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 792/1562] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:13:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:13:14.080293 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cdc01d-8b87-4fe5-963c-04856fa018b4 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:14.080695 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cdc01d-8b87-4fe5-963c-04856fa018b4 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:14.081034 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cdc01d-8b87-4fe5-963c-04856fa018b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:14.104306 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 771/1563] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:13:14 2022] POST /identity/v3/projects => generated 403 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:13:14.104513 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-45cdc01d-8b87-4fe5-963c-04856fa018b4 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:14.104615 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ede92b5-e533-4d0f-adf8-4c3f77877234 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:14.104615 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ede92b5-e533-4d0f-adf8-4c3f77877234 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:14.104615 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ede92b5-e533-4d0f-adf8-4c3f77877234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:14.104902 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-45cdc01d-8b87-4fe5-963c-04856fa018b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:14.123908 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-1ede92b5-e533-4d0f-adf8-4c3f77877234 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:14.125653 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-1ede92b5-e533-4d0f-adf8-4c3f77877234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:14.167588 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 793/1564] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:14.168342 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cdc01d-8b87-4fe5-963c-04856fa018b4 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:14.181276 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cdc01d-8b87-4fe5-963c-04856fa018b4 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:14.186163 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba99e452-ae5e-4ffd-9471-7005ae4d11de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:14.186952 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba99e452-ae5e-4ffd-9471-7005ae4d11de admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:14.187196 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba99e452-ae5e-4ffd-9471-7005ae4d11de admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:14.187477 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba99e452-ae5e-4ffd-9471-7005ae4d11de admin admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:14.188122 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba99e452-ae5e-4ffd-9471-7005ae4d11de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:14.191431 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba99e452-ae5e-4ffd-9471-7005ae4d11de admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:14.199442 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 772/1565] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:14.200605 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586cb2df-1fc9-4486-84fd-1fb3cfe3f82e None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:14.200891 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586cb2df-1fc9-4486-84fd-1fb3cfe3f82e None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:14.201168 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586cb2df-1fc9-4486-84fd-1fb3cfe3f82e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:14.221282 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 794/1566] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:13:14 2022] POST /identity/v3/users => generated 289 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:13:14.221282 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba99e452-ae5e-4ffd-9471-7005ae4d11de admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:14.223438 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-586cb2df-1fc9-4486-84fd-1fb3cfe3f82e None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:14.224901 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-586cb2df-1fc9-4486-84fd-1fb3cfe3f82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:14.229423 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba99e452-ae5e-4ffd-9471-7005ae4d11de admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:14.234538 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02acd027-ba7e-4b9e-8a30-deccb452fab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:14.235011 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02acd027-ba7e-4b9e-8a30-deccb452fab3 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:14.239277 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02acd027-ba7e-4b9e-8a30-deccb452fab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:14.239277 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02acd027-ba7e-4b9e-8a30-deccb452fab3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:14.239575 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 773/1567] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:14.239669 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02acd027-ba7e-4b9e-8a30-deccb452fab3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:14.239759 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586cb2df-1fc9-4486-84fd-1fb3cfe3f82e None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:14.239843 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02acd027-ba7e-4b9e-8a30-deccb452fab3 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:14.250674 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586cb2df-1fc9-4486-84fd-1fb3cfe3f82e None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:14.250885 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 795/1568] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:13:14.251547 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02acd027-ba7e-4b9e-8a30-deccb452fab3 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:14.258464 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a185274-c2c9-487b-9302-408703b0208a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:14.259135 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a185274-c2c9-487b-9302-408703b0208a admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:14.259728 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a185274-c2c9-487b-9302-408703b0208a admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:14.260058 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a185274-c2c9-487b-9302-408703b0208a admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:14.260963 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a185274-c2c9-487b-9302-408703b0208a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:14.263140 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a185274-c2c9-487b-9302-408703b0208a admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:14.265413 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02acd027-ba7e-4b9e-8a30-deccb452fab3 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:14.271617 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6ac39b-2db8-4a13-819f-0188c7a7e5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:14.272241 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6ac39b-2db8-4a13-819f-0188c7a7e5b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:14.272534 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6ac39b-2db8-4a13-819f-0188c7a7e5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:14.272798 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6ac39b-2db8-4a13-819f-0188c7a7e5b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:14.273422 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6ac39b-2db8-4a13-819f-0188c7a7e5b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:14.275322 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6ac39b-2db8-4a13-819f-0188c7a7e5b6 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:14.275707 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 774/1569] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:13:14.276599 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a185274-c2c9-487b-9302-408703b0208a admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:14.288083 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a185274-c2c9-487b-9302-408703b0208a admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:14.292967 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24455d0f-05c6-4bd0-84b8-c9e72a5802d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:14.293555 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24455d0f-05c6-4bd0-84b8-c9e72a5802d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:14.293836 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24455d0f-05c6-4bd0-84b8-c9e72a5802d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:14.294103 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24455d0f-05c6-4bd0-84b8-c9e72a5802d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:14.294720 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24455d0f-05c6-4bd0-84b8-c9e72a5802d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:14.296686 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24455d0f-05c6-4bd0-84b8-c9e72a5802d1 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:14.304707 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 796/1570] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:13:14 2022] POST /identity/v3/projects => generated 365 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:13:14.305529 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6ac39b-2db8-4a13-819f-0188c7a7e5b6 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:14.320194 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 775/1571] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:13:14 2022] POST /identity/v3/users => generated 302 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:13:14.321136 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24455d0f-05c6-4bd0-84b8-c9e72a5802d1 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:14.331275 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24455d0f-05c6-4bd0-84b8-c9e72a5802d1 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:14.369216 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6ac39b-2db8-4a13-819f-0188c7a7e5b6 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:14.373940 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d677e51-1084-492f-9ce5-76c699edf9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:14.374539 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d677e51-1084-492f-9ce5-76c699edf9b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:14.374814 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d677e51-1084-492f-9ce5-76c699edf9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:14.375085 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d677e51-1084-492f-9ce5-76c699edf9b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:14.375726 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d677e51-1084-492f-9ce5-76c699edf9b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:13:14.376023 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d677e51-1084-492f-9ce5-76c699edf9b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:14.378191 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d677e51-1084-492f-9ce5-76c699edf9b6 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:14.388431 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c8e9e-d78f-4074-9987-92f1ec055a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:14.389062 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c8e9e-d78f-4074-9987-92f1ec055a3f admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:14.389347 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c8e9e-d78f-4074-9987-92f1ec055a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:14.389618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c8e9e-d78f-4074-9987-92f1ec055a3f admin admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:14.390209 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 797/1572] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:13:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:13:14.390374 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5c8e9e-d78f-4074-9987-92f1ec055a3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:13:14.390677 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5c8e9e-d78f-4074-9987-92f1ec055a3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:14.391012 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d677e51-1084-492f-9ce5-76c699edf9b6 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:14.392856 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5c8e9e-d78f-4074-9987-92f1ec055a3f admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:14.401379 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d677e51-1084-492f-9ce5-76c699edf9b6 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:14.412502 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 776/1573] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:13:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:13:14.412502 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c8e9e-d78f-4074-9987-92f1ec055a3f admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:14.413137 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48788d86-c99a-43e6-a31b-1b8c5c7c27f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:14.413137 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48788d86-c99a-43e6-a31b-1b8c5c7c27f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:14.413137 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48788d86-c99a-43e6-a31b-1b8c5c7c27f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:14.413137 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48788d86-c99a-43e6-a31b-1b8c5c7c27f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:14.413576 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48788d86-c99a-43e6-a31b-1b8c5c7c27f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:14.413576 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48788d86-c99a-43e6-a31b-1b8c5c7c27f0 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:14.415960 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c8e9e-d78f-4074-9987-92f1ec055a3f admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:14.420961 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb70858-4282-4f79-a1a5-325c3072d907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:14.424977 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 798/1574] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:13:14.425467 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb70858-4282-4f79-a1a5-325c3072d907 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:14.425467 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb70858-4282-4f79-a1a5-325c3072d907 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:14.425467 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb70858-4282-4f79-a1a5-325c3072d907 admin admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:14.425746 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48788d86-c99a-43e6-a31b-1b8c5c7c27f0 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:14.425838 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb70858-4282-4f79-a1a5-325c3072d907 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:13:14.425838 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb70858-4282-4f79-a1a5-325c3072d907 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:14.425838 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb70858-4282-4f79-a1a5-325c3072d907 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:14.433669 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48788d86-c99a-43e6-a31b-1b8c5c7c27f0 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:14.445161 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 777/1575] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:13:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:13:14.445349 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c8f03-bd63-4a4f-8438-e7451e1465e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:14.445468 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb70858-4282-4f79-a1a5-325c3072d907 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:14.445565 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c8f03-bd63-4a4f-8438-e7451e1465e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:14.445565 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c8f03-bd63-4a4f-8438-e7451e1465e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:14.445565 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c8f03-bd63-4a4f-8438-e7451e1465e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:14.445565 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0c8f03-bd63-4a4f-8438-e7451e1465e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:14.445565 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0c8f03-bd63-4a4f-8438-e7451e1465e2 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:14.448299 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:14.450813 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb70858-4282-4f79-a1a5-325c3072d907 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:14.457184 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed916ba5-04e4-4ac5-8aab-3c023be73e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:14.457369 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 799/1576] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:13:14.457549 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed916ba5-04e4-4ac5-8aab-3c023be73e50 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:14.457824 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed916ba5-04e4-4ac5-8aab-3c023be73e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:14.457960 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c8f03-bd63-4a4f-8438-e7451e1465e2 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:14.458093 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed916ba5-04e4-4ac5-8aab-3c023be73e50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:14.458723 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed916ba5-04e4-4ac5-8aab-3c023be73e50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:14.460345 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed916ba5-04e4-4ac5-8aab-3c023be73e50 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:14.467290 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c8f03-bd63-4a4f-8438-e7451e1465e2 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:14.471383 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 778/1577] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:13:14.472122 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7d62b4-73fa-4af7-895a-25838ed3f0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:14.472405 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed916ba5-04e4-4ac5-8aab-3c023be73e50 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:14.472770 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d62b4-73fa-4af7-895a-25838ed3f0d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:14.473005 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d62b4-73fa-4af7-895a-25838ed3f0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:14.473274 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d62b4-73fa-4af7-895a-25838ed3f0d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:14.473900 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7d62b4-73fa-4af7-895a-25838ed3f0d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:14.475814 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7d62b4-73fa-4af7-895a-25838ed3f0d3 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:14.477435 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:14.484023 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed916ba5-04e4-4ac5-8aab-3c023be73e50 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:14.490286 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc10dfc5-0bc4-452b-9408-1c13e9c259ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:14.490890 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10dfc5-0bc4-452b-9408-1c13e9c259ac admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:14.491175 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10dfc5-0bc4-452b-9408-1c13e9c259ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:14.491582 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10dfc5-0bc4-452b-9408-1c13e9c259ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:14.492281 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc10dfc5-0bc4-452b-9408-1c13e9c259ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:14.493957 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc10dfc5-0bc4-452b-9408-1c13e9c259ac admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:14.498380 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 800/1578] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:13:14 2022] POST /identity/v3/users => generated 283 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:13:14.499167 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7d62b4-73fa-4af7-895a-25838ed3f0d3 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:14.507478 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 779/1579] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:13:14.507478 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc10dfc5-0bc4-452b-9408-1c13e9c259ac admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:14.509093 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7d62b4-73fa-4af7-895a-25838ed3f0d3 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:14.514060 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9298bcfa-7de6-4b0d-8fee-7b6b7c20e171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:14.514648 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298bcfa-7de6-4b0d-8fee-7b6b7c20e171 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:14.514925 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298bcfa-7de6-4b0d-8fee-7b6b7c20e171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:14.515056 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc10dfc5-0bc4-452b-9408-1c13e9c259ac admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:14.515191 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298bcfa-7de6-4b0d-8fee-7b6b7c20e171 admin admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:14.515802 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9298bcfa-7de6-4b0d-8fee-7b6b7c20e171 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:14.518553 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9298bcfa-7de6-4b0d-8fee-7b6b7c20e171 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:14.520316 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e4ee38-d1ec-4834-b23d-99e0ba2179a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:14.520893 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e4ee38-d1ec-4834-b23d-99e0ba2179a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:14.521174 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e4ee38-d1ec-4834-b23d-99e0ba2179a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:14.521450 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e4ee38-d1ec-4834-b23d-99e0ba2179a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:14.522093 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e4ee38-d1ec-4834-b23d-99e0ba2179a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:14.523994 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e4ee38-d1ec-4834-b23d-99e0ba2179a1 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:14.546156 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 801/1580] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:13:14 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:13:14.547067 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9298bcfa-7de6-4b0d-8fee-7b6b7c20e171 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:14.554440 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 780/1581] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:13:14 2022] POST /identity/v3/projects => generated 393 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:13:14.554440 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e4ee38-d1ec-4834-b23d-99e0ba2179a1 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:14.610553 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9298bcfa-7de6-4b0d-8fee-7b6b7c20e171 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:14.614189 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e4ee38-d1ec-4834-b23d-99e0ba2179a1 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:14.616740 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684c5599-caec-4049-a1bc-56e4c01e36c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:14.617352 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684c5599-caec-4049-a1bc-56e4c01e36c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:14.617629 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684c5599-caec-4049-a1bc-56e4c01e36c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:14.617896 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684c5599-caec-4049-a1bc-56e4c01e36c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:14.618532 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684c5599-caec-4049-a1bc-56e4c01e36c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:14.620151 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684c5599-caec-4049-a1bc-56e4c01e36c3 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:14.624427 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70e443-9890-46f8-8399-63f936484566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:14.625048 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70e443-9890-46f8-8399-63f936484566 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:14.625327 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70e443-9890-46f8-8399-63f936484566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:14.625600 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70e443-9890-46f8-8399-63f936484566 admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:14.626232 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b70e443-9890-46f8-8399-63f936484566 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:14.630560 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b70e443-9890-46f8-8399-63f936484566 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:14.630720 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 802/1582] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:13:14.631367 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684c5599-caec-4049-a1bc-56e4c01e36c3 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:14.641649 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684c5599-caec-4049-a1bc-56e4c01e36c3 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:14.647268 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f76534-3dae-42e5-ad49-56c3692fc037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:14.648012 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f76534-3dae-42e5-ad49-56c3692fc037 admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:14.648386 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f76534-3dae-42e5-ad49-56c3692fc037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:14.648705 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f76534-3dae-42e5-ad49-56c3692fc037 admin admin] PATH_INFO: `/v3/projects/f33f4d95a2ba4eb693655b23f63fbe2e/users/ad38d50e318441698ff2820858e8d499/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:14.660397 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 781/1583] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:13:14 2022] POST /identity/v3/projects => generated 375 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:13:14.660397 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70e443-9890-46f8-8399-63f936484566 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:14.664841 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f76534-3dae-42e5-ad49-56c3692fc037 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f33f4d95a2ba4eb693655b23f63fbe2e, user_id=ad38d50e318441698ff2820858e8d499, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:14.667506 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f76534-3dae-42e5-ad49-56c3692fc037 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:14.682191 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 803/1584] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:13:14 2022] PUT /identity/v3/projects/f33f4d95a2ba4eb693655b23f63fbe2e/users/ad38d50e318441698ff2820858e8d499/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:14.682918 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f76534-3dae-42e5-ad49-56c3692fc037 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:14.723923 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70e443-9890-46f8-8399-63f936484566 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:14.751608 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f76534-3dae-42e5-ad49-56c3692fc037 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:14.756347 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721faf47-0bac-4869-a007-f525f1d32087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:14.756956 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721faf47-0bac-4869-a007-f525f1d32087 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:14.757238 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721faf47-0bac-4869-a007-f525f1d32087 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:14.757514 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721faf47-0bac-4869-a007-f525f1d32087 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:14.758157 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721faf47-0bac-4869-a007-f525f1d32087 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:14.760049 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721faf47-0bac-4869-a007-f525f1d32087 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:14.770369 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 804/1585] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:13:14.771260 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721faf47-0bac-4869-a007-f525f1d32087 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:14.784942 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721faf47-0bac-4869-a007-f525f1d32087 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:14.789719 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a7c48e-93db-43b2-9bc6-e026d6b4a4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:14.789942 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088eb400-cd67-4fab-9a38-f48e579ea544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:14.790399 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a7c48e-93db-43b2-9bc6-e026d6b4a4c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:14.790565 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088eb400-cd67-4fab-9a38-f48e579ea544 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:14.790686 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a7c48e-93db-43b2-9bc6-e026d6b4a4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:14.790840 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088eb400-cd67-4fab-9a38-f48e579ea544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:14.790953 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a7c48e-93db-43b2-9bc6-e026d6b4a4c8 admin admin] PATH_INFO: `/v3/projects/34623908736a4e3d9f015fc9db90bd0d/users/9f2e208a67d4443eb00c75c60c54173f/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:14.791108 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088eb400-cd67-4fab-9a38-f48e579ea544 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:14.791767 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088eb400-cd67-4fab-9a38-f48e579ea544 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:14.793445 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088eb400-cd67-4fab-9a38-f48e579ea544 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:14.794884 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:14.803804 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 805/1586] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:13:14.804607 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088eb400-cd67-4fab-9a38-f48e579ea544 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:14.808721 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a7c48e-93db-43b2-9bc6-e026d6b4a4c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34623908736a4e3d9f015fc9db90bd0d, user_id=9f2e208a67d4443eb00c75c60c54173f, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:14.809778 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: WARNING py.warnings [None req-b7a7c48e-93db-43b2-9bc6-e026d6b4a4c8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 01:13:14.809778 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: warnings.warn(msg) Sep 08 01:13:14.809778 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: Sep 08 01:13:14.811726 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a7c48e-93db-43b2-9bc6-e026d6b4a4c8 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:14.813960 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088eb400-cd67-4fab-9a38-f48e579ea544 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:14.819745 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc03fc6f-148c-4e26-a2de-2d58edd729f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:14.820378 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03fc6f-148c-4e26-a2de-2d58edd729f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:14.820664 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03fc6f-148c-4e26-a2de-2d58edd729f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:14.820991 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03fc6f-148c-4e26-a2de-2d58edd729f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:14.821541 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc03fc6f-148c-4e26-a2de-2d58edd729f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:14.823431 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc03fc6f-148c-4e26-a2de-2d58edd729f6 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:14.826045 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 782/1587] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:13:14 2022] PUT /identity/v3/projects/34623908736a4e3d9f015fc9db90bd0d/users/9f2e208a67d4443eb00c75c60c54173f/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:14.827110 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a7c48e-93db-43b2-9bc6-e026d6b4a4c8 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:14.842329 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:14.852508 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 806/1588] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:13:14 2022] POST /identity/v3/users => generated 291 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:13:14.853350 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc03fc6f-148c-4e26-a2de-2d58edd729f6 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:14.863670 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc03fc6f-148c-4e26-a2de-2d58edd729f6 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:14.891865 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a7c48e-93db-43b2-9bc6-e026d6b4a4c8 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:14.897999 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef57767b-1bec-49cb-bbe1-c1c6bfe1e770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:14.898606 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57767b-1bec-49cb-bbe1-c1c6bfe1e770 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:14.898882 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57767b-1bec-49cb-bbe1-c1c6bfe1e770 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:14.899151 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57767b-1bec-49cb-bbe1-c1c6bfe1e770 admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:14.899772 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef57767b-1bec-49cb-bbe1-c1c6bfe1e770 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:14.901716 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef57767b-1bec-49cb-bbe1-c1c6bfe1e770 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:14.921931 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8078c19d-5734-4d43-9905-cf45e0e1bf4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:14.922595 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078c19d-5734-4d43-9905-cf45e0e1bf4d admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:14.922868 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078c19d-5734-4d43-9905-cf45e0e1bf4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:14.923135 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078c19d-5734-4d43-9905-cf45e0e1bf4d admin admin] PATH_INFO: `/v3/projects/c1186c0555f345b6915c81b1f33b84bc/users/36fe14a406444624a34a73e2ad60a75c/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:14.926995 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 783/1589] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:13:14 2022] POST /identity/v3/users => generated 297 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:13:14.926995 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef57767b-1bec-49cb-bbe1-c1c6bfe1e770 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:14.934559 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef57767b-1bec-49cb-bbe1-c1c6bfe1e770 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:14.938199 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8078c19d-5734-4d43-9905-cf45e0e1bf4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1186c0555f345b6915c81b1f33b84bc, user_id=36fe14a406444624a34a73e2ad60a75c, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:14.939653 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadca58d-7e4c-4c56-910e-68f6030a9e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:14.941482 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadca58d-7e4c-4c56-910e-68f6030a9e8e admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:14.941769 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadca58d-7e4c-4c56-910e-68f6030a9e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:14.942040 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadca58d-7e4c-4c56-910e-68f6030a9e8e admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:14.942659 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadca58d-7e4c-4c56-910e-68f6030a9e8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:14.942771 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8078c19d-5734-4d43-9905-cf45e0e1bf4d admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:14.944597 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadca58d-7e4c-4c56-910e-68f6030a9e8e admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:14.959771 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 807/1590] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:13:14 2022] PUT /identity/v3/projects/c1186c0555f345b6915c81b1f33b84bc/users/36fe14a406444624a34a73e2ad60a75c/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:14.959771 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72649d-8125-4bfd-946e-ac51fcfbf78e None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:14.959771 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72649d-8125-4bfd-946e-ac51fcfbf78e None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:14.959771 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72649d-8125-4bfd-946e-ac51fcfbf78e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:14.975225 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 784/1591] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:13:14 2022] POST /identity/v3/users => generated 288 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:13:14.975225 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadca58d-7e4c-4c56-910e-68f6030a9e8e admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:14.982853 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadca58d-7e4c-4c56-910e-68f6030a9e8e admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:14.989173 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-1f72649d-8125-4bfd-946e-ac51fcfbf78e None None] MFA Rules not processed for user `ad38d50e318441698ff2820858e8d499`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:14.990439 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-1f72649d-8125-4bfd-946e-ac51fcfbf78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:15.039260 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373813e1-df4c-467a-9454-0a87081f48fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:15.040123 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373813e1-df4c-467a-9454-0a87081f48fa admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:15.040444 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373813e1-df4c-467a-9454-0a87081f48fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:15.040722 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373813e1-df4c-467a-9454-0a87081f48fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:15.041430 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373813e1-df4c-467a-9454-0a87081f48fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:15.043050 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373813e1-df4c-467a-9454-0a87081f48fa admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:15.053077 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 785/1592] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:13:15.053854 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373813e1-df4c-467a-9454-0a87081f48fa admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:15.057662 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 808/1593] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:14 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:15.059052 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe0cb3-560b-44b1-ae55-7cfca7f25734 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:15.059341 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe0cb3-560b-44b1-ae55-7cfca7f25734 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:15.059615 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe0cb3-560b-44b1-ae55-7cfca7f25734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:15.066852 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373813e1-df4c-467a-9454-0a87081f48fa admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:15.072094 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c6b1a8-9a4b-4612-9a75-c8f139b357ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:15.072790 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c6b1a8-9a4b-4612-9a75-c8f139b357ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:15.073070 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c6b1a8-9a4b-4612-9a75-c8f139b357ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:15.073346 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c6b1a8-9a4b-4612-9a75-c8f139b357ec admin admin] PATH_INFO: `/v3/projects/8eed1dc3232042248cd4adeaefdcc910/users/86f94747a92b4c3a8bbf2aad7d912758/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:15.079364 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-4ffe0cb3-560b-44b1-ae55-7cfca7f25734 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:15.081170 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-4ffe0cb3-560b-44b1-ae55-7cfca7f25734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:15.088835 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c6b1a8-9a4b-4612-9a75-c8f139b357ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=8eed1dc3232042248cd4adeaefdcc910, user_id=86f94747a92b4c3a8bbf2aad7d912758, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:15.088981 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 809/1594] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:15.091523 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c6b1a8-9a4b-4612-9a75-c8f139b357ec admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:15.092946 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe0cb3-560b-44b1-ae55-7cfca7f25734 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:15.103043 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe0cb3-560b-44b1-ae55-7cfca7f25734 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:15.106332 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 786/1595] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:13:15 2022] PUT /identity/v3/projects/8eed1dc3232042248cd4adeaefdcc910/users/86f94747a92b4c3a8bbf2aad7d912758/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:15.107272 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c6b1a8-9a4b-4612-9a75-c8f139b357ec admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:15.139992 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:15.167863 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170003a4-efef-46ff-bec8-a0202f34803d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:15.168496 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170003a4-efef-46ff-bec8-a0202f34803d admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:15.168777 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170003a4-efef-46ff-bec8-a0202f34803d admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:15.169046 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170003a4-efef-46ff-bec8-a0202f34803d admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:15.169675 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170003a4-efef-46ff-bec8-a0202f34803d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:15.171265 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170003a4-efef-46ff-bec8-a0202f34803d admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:15.172423 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c6b1a8-9a4b-4612-9a75-c8f139b357ec admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:15.177754 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7f578f-3b67-4751-8792-8f582f88d34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:15.178360 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7f578f-3b67-4751-8792-8f582f88d34f admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:15.178643 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7f578f-3b67-4751-8792-8f582f88d34f admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:15.178918 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7f578f-3b67-4751-8792-8f582f88d34f admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:15.179557 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7f578f-3b67-4751-8792-8f582f88d34f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:15.181193 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7f578f-3b67-4751-8792-8f582f88d34f admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:15.181692 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 810/1596] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:13:15 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:13:15.185863 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca88863c-fa59-4a8e-8c50-3fbbc67cfdd6 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:15.186104 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca88863c-fa59-4a8e-8c50-3fbbc67cfdd6 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:15.186374 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca88863c-fa59-4a8e-8c50-3fbbc67cfdd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:15.191727 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 787/1597] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:13:15 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:13:15.192592 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7f578f-3b67-4751-8792-8f582f88d34f admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:15.193673 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:15.204085 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7f578f-3b67-4751-8792-8f582f88d34f admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:15.209917 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-ca88863c-fa59-4a8e-8c50-3fbbc67cfdd6 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:15.210084 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000caf4b-2d68-4316-b5d4-e2a35ff2fdc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:15.210521 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000caf4b-2d68-4316-b5d4-e2a35ff2fdc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:15.210803 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000caf4b-2d68-4316-b5d4-e2a35ff2fdc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:15.211076 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000caf4b-2d68-4316-b5d4-e2a35ff2fdc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:15.211181 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-ca88863c-fa59-4a8e-8c50-3fbbc67cfdd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:15.211715 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000caf4b-2d68-4316-b5d4-e2a35ff2fdc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:15.213351 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000caf4b-2d68-4316-b5d4-e2a35ff2fdc8 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:15.219416 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 811/1598] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:15.220385 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca88863c-fa59-4a8e-8c50-3fbbc67cfdd6 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:15.227784 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 788/1599] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:13:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:13:15.227784 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000caf4b-2d68-4316-b5d4-e2a35ff2fdc8 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:15.231911 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca88863c-fa59-4a8e-8c50-3fbbc67cfdd6 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:15.236937 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb0ce8c-e82d-43b1-badc-2265b9d0db2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:15.237093 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000caf4b-2d68-4316-b5d4-e2a35ff2fdc8 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:15.237601 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0ce8c-e82d-43b1-badc-2265b9d0db2f admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:15.237879 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0ce8c-e82d-43b1-badc-2265b9d0db2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:15.238156 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0ce8c-e82d-43b1-badc-2265b9d0db2f admin admin] PATH_INFO: `/v3/projects/18b694774b5b4608a768e77415dfcd02/users/11b818b09de243da96959a412ee92075/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:15.241945 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e29583d-6245-465b-85b6-679a432c666c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:15.242533 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e29583d-6245-465b-85b6-679a432c666c admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:15.242811 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e29583d-6245-465b-85b6-679a432c666c admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:15.243081 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e29583d-6245-465b-85b6-679a432c666c admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:15.243698 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e29583d-6245-465b-85b6-679a432c666c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:15.245644 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e29583d-6245-465b-85b6-679a432c666c admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:15.253622 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb0ce8c-e82d-43b1-badc-2265b9d0db2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=18b694774b5b4608a768e77415dfcd02, user_id=11b818b09de243da96959a412ee92075, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:15.256691 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb0ce8c-e82d-43b1-badc-2265b9d0db2f admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:15.271575 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 812/1600] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:13:15 2022] PUT /identity/v3/projects/18b694774b5b4608a768e77415dfcd02/users/11b818b09de243da96959a412ee92075/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:15.272421 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb0ce8c-e82d-43b1-badc-2265b9d0db2f admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:15.277612 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 789/1601] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:13:15 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:13:15.277612 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ec280-f3ea-4ca2-a030-00463c878ced None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:15.277612 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ec280-f3ea-4ca2-a030-00463c878ced None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:15.277612 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ec280-f3ea-4ca2-a030-00463c878ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:15.305818 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-3d9ec280-f3ea-4ca2-a030-00463c878ced None None] MFA Rules not processed for user `86f94747a92b4c3a8bbf2aad7d912758`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:15.307037 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-3d9ec280-f3ea-4ca2-a030-00463c878ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:15.333174 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb0ce8c-e82d-43b1-badc-2265b9d0db2f admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:15.373775 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 790/1602] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:15 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:15.374661 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9ec280-f3ea-4ca2-a030-00463c878ced None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:15.383229 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c9a07b-7b76-491f-954c-0788e260edb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:15.383843 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c9a07b-7b76-491f-954c-0788e260edb4 service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:15.384156 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c9a07b-7b76-491f-954c-0788e260edb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:15.384703 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c9a07b-7b76-491f-954c-0788e260edb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:15.436219 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9ec280-f3ea-4ca2-a030-00463c878ced None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:15.441476 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa911fba-c444-4636-9f1a-9cce646cd199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:15.442376 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa911fba-c444-4636-9f1a-9cce646cd199 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:15.442376 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa911fba-c444-4636-9f1a-9cce646cd199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:15.442629 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa911fba-c444-4636-9f1a-9cce646cd199 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:15.443276 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa911fba-c444-4636-9f1a-9cce646cd199 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:15.444940 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa911fba-c444-4636-9f1a-9cce646cd199 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:15.448331 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c9a07b-7b76-491f-954c-0788e260edb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:15.449945 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c9a07b-7b76-491f-954c-0788e260edb4 service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:15.459375 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 791/1603] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:13:15 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:13:15.459375 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa911fba-c444-4636-9f1a-9cce646cd199 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:15.461321 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 813/1604] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:13:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:13:15.462136 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c9a07b-7b76-491f-954c-0788e260edb4 service neutron] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:15.468777 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa911fba-c444-4636-9f1a-9cce646cd199 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:15.472217 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c9a07b-7b76-491f-954c-0788e260edb4 service neutron] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:15.474720 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58ca87e-db45-4392-a979-abc5d0c00d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:15.475465 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ca87e-db45-4392-a979-abc5d0c00d8f admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:15.475847 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ca87e-db45-4392-a979-abc5d0c00d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:15.476304 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ca87e-db45-4392-a979-abc5d0c00d8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:15.477011 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97af621-9f77-4bf7-a53e-282e3fc74153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:15.477162 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58ca87e-db45-4392-a979-abc5d0c00d8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:15.477652 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97af621-9f77-4bf7-a53e-282e3fc74153 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:15.477933 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97af621-9f77-4bf7-a53e-282e3fc74153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:15.478208 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97af621-9f77-4bf7-a53e-282e3fc74153 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:15.478886 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97af621-9f77-4bf7-a53e-282e3fc74153 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:15.479371 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58ca87e-db45-4392-a979-abc5d0c00d8f admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:15.480561 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97af621-9f77-4bf7-a53e-282e3fc74153 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:15.491677 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 814/1605] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:13:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:13:15.495632 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 792/1606] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:13:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:13:15.496097 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97af621-9f77-4bf7-a53e-282e3fc74153 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:15.496259 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58ca87e-db45-4392-a979-abc5d0c00d8f admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:15.506107 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97af621-9f77-4bf7-a53e-282e3fc74153 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:15.507070 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58ca87e-db45-4392-a979-abc5d0c00d8f admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:15.511183 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f68bd2-738a-4f82-ab0a-48d59d27b479 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:15.511960 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f68bd2-738a-4f82-ab0a-48d59d27b479 service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:15.512336 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f68bd2-738a-4f82-ab0a-48d59d27b479 service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:15.512472 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf5b4da-8623-4a7b-9b82-08642cb0fcd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:15.512735 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f68bd2-738a-4f82-ab0a-48d59d27b479 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:15.513193 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf5b4da-8623-4a7b-9b82-08642cb0fcd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:15.513533 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf5b4da-8623-4a7b-9b82-08642cb0fcd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:15.513875 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf5b4da-8623-4a7b-9b82-08642cb0fcd7 admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:15.514577 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf5b4da-8623-4a7b-9b82-08642cb0fcd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:15.516652 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf5b4da-8623-4a7b-9b82-08642cb0fcd7 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:15.524227 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f68bd2-738a-4f82-ab0a-48d59d27b479 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:15.525695 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f68bd2-738a-4f82-ab0a-48d59d27b479 service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:15.526138 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:15.537294 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 815/1607] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:13:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:13:15.538489 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6dc72-c88b-419c-8fd6-0988d467a9c0 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:15.538783 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6dc72-c88b-419c-8fd6-0988d467a9c0 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:15.539061 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6dc72-c88b-419c-8fd6-0988d467a9c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:15.545394 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 793/1608] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:13:15 2022] POST /identity/v3/users => generated 300 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:13:15.546700 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf5b4da-8623-4a7b-9b82-08642cb0fcd7 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:15.557971 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf5b4da-8623-4a7b-9b82-08642cb0fcd7 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:15.563338 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b22aa2f-df4c-4551-810c-5356081267f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:15.564268 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22aa2f-df4c-4551-810c-5356081267f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:15.564842 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22aa2f-df4c-4551-810c-5356081267f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:15.565200 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22aa2f-df4c-4551-810c-5356081267f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:15.566184 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b22aa2f-df4c-4551-810c-5356081267f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:15.568005 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b22aa2f-df4c-4551-810c-5356081267f3 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:15.571748 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:15.575594 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-8ee6dc72-c88b-419c-8fd6-0988d467a9c0 None None] MFA Rules not processed for user `11b818b09de243da96959a412ee92075`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:15.577909 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-8ee6dc72-c88b-419c-8fd6-0988d467a9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:15.598885 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 794/1609] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:13:15 2022] POST /identity/v3/projects => generated 403 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:13:15.599525 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b22aa2f-df4c-4551-810c-5356081267f3 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:15.668344 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 816/1610] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:15 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:15.669051 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee6dc72-c88b-419c-8fd6-0988d467a9c0 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:15.672986 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b22aa2f-df4c-4551-810c-5356081267f3 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:15.678363 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a5003c-83fb-46cf-a600-a063990fef2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:15.679043 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5003c-83fb-46cf-a600-a063990fef2d admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:15.679307 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5003c-83fb-46cf-a600-a063990fef2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:15.679581 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5003c-83fb-46cf-a600-a063990fef2d admin admin] PATH_INFO: `/v3/projects/c8afdac5e54e44eaadb045a4ca92bf21/users/cbeec441c6154c81a9465a2955f7c8aa/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:15.681666 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee6dc72-c88b-419c-8fd6-0988d467a9c0 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:15.688464 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2499ed4b-0d49-4f6b-8e9d-42843f8b9635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:15.689595 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2499ed4b-0d49-4f6b-8e9d-42843f8b9635 admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:15.689881 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2499ed4b-0d49-4f6b-8e9d-42843f8b9635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:15.690211 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2499ed4b-0d49-4f6b-8e9d-42843f8b9635 admin admin] PATH_INFO: `/v3/projects/8c24dd087c274b6094aa720d81b11e98/users/3246a8dea80d4d5b8c3fb50d5f1f31c0/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:15.699000 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a5003c-83fb-46cf-a600-a063990fef2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8afdac5e54e44eaadb045a4ca92bf21, user_id=cbeec441c6154c81a9465a2955f7c8aa, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:15.701719 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a5003c-83fb-46cf-a600-a063990fef2d admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:15.708666 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2499ed4b-0d49-4f6b-8e9d-42843f8b9635 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c24dd087c274b6094aa720d81b11e98, user_id=3246a8dea80d4d5b8c3fb50d5f1f31c0, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:15.711363 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2499ed4b-0d49-4f6b-8e9d-42843f8b9635 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:15.718898 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 795/1611] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:13:15 2022] PUT /identity/v3/projects/c8afdac5e54e44eaadb045a4ca92bf21/users/cbeec441c6154c81a9465a2955f7c8aa/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:15.719980 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a5003c-83fb-46cf-a600-a063990fef2d admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:15.739278 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 817/1612] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:13:15 2022] PUT /identity/v3/projects/8c24dd087c274b6094aa720d81b11e98/users/3246a8dea80d4d5b8c3fb50d5f1f31c0/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:15.740229 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2499ed4b-0d49-4f6b-8e9d-42843f8b9635 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:15.807208 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a5003c-83fb-46cf-a600-a063990fef2d admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:15.812549 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f9cfb-eaf8-42f9-a1ac-51247dc9b5c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:15.813219 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f9cfb-eaf8-42f9-a1ac-51247dc9b5c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:15.813500 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f9cfb-eaf8-42f9-a1ac-51247dc9b5c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:15.813778 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f9cfb-eaf8-42f9-a1ac-51247dc9b5c5 admin admin] PATH_INFO: `/v3/projects/f520f61ec5304a72ac707318d3a73b4f/users/20af4425b7434d2dab7baeacbb845387/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:15.815795 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2499ed4b-0d49-4f6b-8e9d-42843f8b9635 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:15.820657 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0de713f-9efc-4b19-b3a6-13d95a812f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:15.821272 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de713f-9efc-4b19-b3a6-13d95a812f8b admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:15.821549 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de713f-9efc-4b19-b3a6-13d95a812f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:15.822008 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de713f-9efc-4b19-b3a6-13d95a812f8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:15.822764 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0de713f-9efc-4b19-b3a6-13d95a812f8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:15.824069 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0de713f-9efc-4b19-b3a6-13d95a812f8b admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:15.833241 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339f9cfb-eaf8-42f9-a1ac-51247dc9b5c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f520f61ec5304a72ac707318d3a73b4f, user_id=20af4425b7434d2dab7baeacbb845387, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:15.838882 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 818/1613] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:13:15 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:13:15.839035 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339f9cfb-eaf8-42f9-a1ac-51247dc9b5c5 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:15.839118 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0de713f-9efc-4b19-b3a6-13d95a812f8b admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:15.855065 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0de713f-9efc-4b19-b3a6-13d95a812f8b admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:15.864937 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 796/1614] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:13:15 2022] PUT /identity/v3/projects/f520f61ec5304a72ac707318d3a73b4f/users/20af4425b7434d2dab7baeacbb845387/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:15.872968 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f9cfb-eaf8-42f9-a1ac-51247dc9b5c5 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:15.912450 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:15.924612 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aebf12-c8ab-4eb7-b84e-3109bea80029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:15.925594 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aebf12-c8ab-4eb7-b84e-3109bea80029 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:15.925886 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aebf12-c8ab-4eb7-b84e-3109bea80029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:15.926276 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aebf12-c8ab-4eb7-b84e-3109bea80029 admin admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:15.926803 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aebf12-c8ab-4eb7-b84e-3109bea80029 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:15.929083 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aebf12-c8ab-4eb7-b84e-3109bea80029 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:15.952614 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f9cfb-eaf8-42f9-a1ac-51247dc9b5c5 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:15.962965 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 819/1615] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:13:15 2022] POST /identity/v3/users => generated 308 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:13:15.962965 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2181dae8-ca57-41ea-978a-f1a30faa792c None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:15.962965 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2181dae8-ca57-41ea-978a-f1a30faa792c None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:15.962965 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2181dae8-ca57-41ea-978a-f1a30faa792c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:15.965640 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1ef3ed-0e29-42e6-a7f1-43169395e45e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:15.966802 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ef3ed-0e29-42e6-a7f1-43169395e45e admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:15.966802 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ef3ed-0e29-42e6-a7f1-43169395e45e admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:15.967035 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ef3ed-0e29-42e6-a7f1-43169395e45e admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:15.967523 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1ef3ed-0e29-42e6-a7f1-43169395e45e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:15.970447 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1ef3ed-0e29-42e6-a7f1-43169395e45e admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:15.974345 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:16.001045 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-2181dae8-ca57-41ea-978a-f1a30faa792c None None] MFA Rules not processed for user `cbeec441c6154c81a9465a2955f7c8aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:16.001462 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 797/1616] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:13:15 2022] POST /identity/v3/projects => generated 365 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:13:16.002923 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5fac79-8933-4a46-87e1-07c735ebe103 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:16.003209 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5fac79-8933-4a46-87e1-07c735ebe103 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:16.003480 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5fac79-8933-4a46-87e1-07c735ebe103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:16.005910 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-2181dae8-ca57-41ea-978a-f1a30faa792c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:16.023277 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-ec5fac79-8933-4a46-87e1-07c735ebe103 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:16.024513 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-ec5fac79-8933-4a46-87e1-07c735ebe103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:16.098309 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 820/1617] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:15 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:16.099245 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2181dae8-ca57-41ea-978a-f1a30faa792c None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:16.108051 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 798/1618] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:16.112966 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2181dae8-ca57-41ea-978a-f1a30faa792c None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:16.113806 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad492f7d-394d-4541-989d-2e63ca1cb46e None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:16.114101 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad492f7d-394d-4541-989d-2e63ca1cb46e None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:16.114378 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad492f7d-394d-4541-989d-2e63ca1cb46e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:16.118692 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2349aa22-5cc1-4f26-b1e3-60ca37fab938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:16.119723 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2349aa22-5cc1-4f26-b1e3-60ca37fab938 admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:16.119928 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2349aa22-5cc1-4f26-b1e3-60ca37fab938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:16.120163 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2349aa22-5cc1-4f26-b1e3-60ca37fab938 admin admin] PATH_INFO: `/v3/projects/031ebc86f61d41018bec3c8abf53b26e/users/1f864c06f7ef46fd9211899da7cec39a/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:16.144425 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-ad492f7d-394d-4541-989d-2e63ca1cb46e None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:16.146337 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-ad492f7d-394d-4541-989d-2e63ca1cb46e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:16.148927 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2349aa22-5cc1-4f26-b1e3-60ca37fab938 admin admin] RBAC: Authorizing `identity:create_grant(project_id=031ebc86f61d41018bec3c8abf53b26e, user_id=1f864c06f7ef46fd9211899da7cec39a, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:16.155652 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2349aa22-5cc1-4f26-b1e3-60ca37fab938 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:16.160332 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 799/1619] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:16.160332 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad492f7d-394d-4541-989d-2e63ca1cb46e None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:16.177344 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 821/1620] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:13:16 2022] PUT /identity/v3/projects/031ebc86f61d41018bec3c8abf53b26e/users/1f864c06f7ef46fd9211899da7cec39a/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:16.178243 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad492f7d-394d-4541-989d-2e63ca1cb46e None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:16.178597 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2349aa22-5cc1-4f26-b1e3-60ca37fab938 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:16.268456 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d08300-22bf-436a-aad1-269831acc95e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:16.269394 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d08300-22bf-436a-aad1-269831acc95e admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:16.269834 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d08300-22bf-436a-aad1-269831acc95e admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:16.270266 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d08300-22bf-436a-aad1-269831acc95e admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:16.271245 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2349aa22-5cc1-4f26-b1e3-60ca37fab938 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:16.271404 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d08300-22bf-436a-aad1-269831acc95e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:16.273877 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d08300-22bf-436a-aad1-269831acc95e admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:16.276973 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635d1ef6-9111-4ab4-9fe3-1032b9099dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:16.277745 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d1ef6-9111-4ab4-9fe3-1032b9099dfa admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:16.278213 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d1ef6-9111-4ab4-9fe3-1032b9099dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:16.278608 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d1ef6-9111-4ab4-9fe3-1032b9099dfa admin admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:16.279508 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635d1ef6-9111-4ab4-9fe3-1032b9099dfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:16.281792 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635d1ef6-9111-4ab4-9fe3-1032b9099dfa admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:16.294418 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 800/1621] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:13:16 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:13:16.296355 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d08300-22bf-436a-aad1-269831acc95e admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:16.310622 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 822/1622] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:13:16 2022] POST /identity/v3/users => generated 289 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:13:16.313378 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635d1ef6-9111-4ab4-9fe3-1032b9099dfa admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:16.316625 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-635d1ef6-9111-4ab4-9fe3-1032b9099dfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:16.317014 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d08300-22bf-436a-aad1-269831acc95e admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:16.323507 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9831f511-4251-401b-8c50-3bafb1a15cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:16.324254 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831f511-4251-401b-8c50-3bafb1a15cca admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:16.324713 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831f511-4251-401b-8c50-3bafb1a15cca admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:16.325267 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831f511-4251-401b-8c50-3bafb1a15cca admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:16.325987 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9831f511-4251-401b-8c50-3bafb1a15cca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:16.328709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9831f511-4251-401b-8c50-3bafb1a15cca admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:16.366813 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 801/1623] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:13:16 2022] POST /identity/v3/projects => generated 379 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:13:16.366813 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9831f511-4251-401b-8c50-3bafb1a15cca admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:16.399092 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635d1ef6-9111-4ab4-9fe3-1032b9099dfa admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:16.409726 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:16.418642 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:16.462808 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9831f511-4251-401b-8c50-3bafb1a15cca admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:16.469440 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aff9ba-28d4-41cc-9bab-bdcc4aebc0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:16.470845 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aff9ba-28d4-41cc-9bab-bdcc4aebc0ad admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:16.471210 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aff9ba-28d4-41cc-9bab-bdcc4aebc0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:16.471570 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aff9ba-28d4-41cc-9bab-bdcc4aebc0ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:16.472486 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aff9ba-28d4-41cc-9bab-bdcc4aebc0ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:16.474304 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aff9ba-28d4-41cc-9bab-bdcc4aebc0ad admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:16.476734 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c243df46-b645-450b-a404-3f70b75b98ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:16.477746 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c243df46-b645-450b-a404-3f70b75b98ca service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:16.478065 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c243df46-b645-450b-a404-3f70b75b98ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:16.478343 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c243df46-b645-450b-a404-3f70b75b98ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:16.488111 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 802/1624] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:13:16 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:13:16.488901 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aff9ba-28d4-41cc-9bab-bdcc4aebc0ad admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:16.494025 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c243df46-b645-450b-a404-3f70b75b98ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:16.496810 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c243df46-b645-450b-a404-3f70b75b98ca service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:16.510565 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 823/1625] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:13:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:13:16.511537 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c243df46-b645-450b-a404-3f70b75b98ca service neutron] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:16.513985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aff9ba-28d4-41cc-9bab-bdcc4aebc0ad admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:16.521610 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94466cfb-d0aa-4caa-b89e-6f2eeec7bace service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:16.522387 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94466cfb-d0aa-4caa-b89e-6f2eeec7bace service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:16.522777 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94466cfb-d0aa-4caa-b89e-6f2eeec7bace service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:16.523155 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94466cfb-d0aa-4caa-b89e-6f2eeec7bace service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:16.528923 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c243df46-b645-450b-a404-3f70b75b98ca service neutron] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:16.536776 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94466cfb-d0aa-4caa-b89e-6f2eeec7bace service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:16.537520 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e9d2d-62cb-4322-88a5-1dde051b06a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:16.538495 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e9d2d-62cb-4322-88a5-1dde051b06a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:16.538724 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e9d2d-62cb-4322-88a5-1dde051b06a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:16.539046 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e9d2d-62cb-4322-88a5-1dde051b06a5 admin admin] PATH_INFO: `/v3/projects/1a5ba109e4ed45cea2cde2ae3b3dff13/users/ad899bf9479d4270ba8c9d35258282a3/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:16.539476 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94466cfb-d0aa-4caa-b89e-6f2eeec7bace service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:16.551852 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 803/1626] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:13:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:13:16.552919 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94466cfb-d0aa-4caa-b89e-6f2eeec7bace service neutron] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:16.559122 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617e9d2d-62cb-4322-88a5-1dde051b06a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a5ba109e4ed45cea2cde2ae3b3dff13, user_id=ad899bf9479d4270ba8c9d35258282a3, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:16.564281 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617e9d2d-62cb-4322-88a5-1dde051b06a5 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:16.565226 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94466cfb-d0aa-4caa-b89e-6f2eeec7bace service neutron] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:16.571728 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93195f55-bd2a-4843-936c-22a8ce839d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:16.572587 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93195f55-bd2a-4843-936c-22a8ce839d0d admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:16.573060 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93195f55-bd2a-4843-936c-22a8ce839d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:16.573348 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93195f55-bd2a-4843-936c-22a8ce839d0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:16.574193 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93195f55-bd2a-4843-936c-22a8ce839d0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:16.576036 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93195f55-bd2a-4843-936c-22a8ce839d0d admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:16.593058 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 804/1627] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:13:16 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:13:16.593983 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93195f55-bd2a-4843-936c-22a8ce839d0d admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:16.599031 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 824/1628] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:13:16 2022] PUT /identity/v3/projects/1a5ba109e4ed45cea2cde2ae3b3dff13/users/ad899bf9479d4270ba8c9d35258282a3/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:16.601773 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e9d2d-62cb-4322-88a5-1dde051b06a5 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:16.605881 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93195f55-bd2a-4843-936c-22a8ce839d0d admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:16.686448 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f8b498-9de3-4089-8b88-62184f256630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:16.687719 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f8b498-9de3-4089-8b88-62184f256630 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:16.687921 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f8b498-9de3-4089-8b88-62184f256630 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:16.688569 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f8b498-9de3-4089-8b88-62184f256630 admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:16.689332 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f8b498-9de3-4089-8b88-62184f256630 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:16.691767 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f8b498-9de3-4089-8b88-62184f256630 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:16.735395 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 805/1629] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:13:16 2022] POST /identity/v3/users => generated 296 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:13:16.737121 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f8b498-9de3-4089-8b88-62184f256630 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:16.747388 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e9d2d-62cb-4322-88a5-1dde051b06a5 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:16.756678 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f8b498-9de3-4089-8b88-62184f256630 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:16.760797 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc84f1ba-6f3c-44b4-a270-d3fcc3b15175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:16.761611 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc84f1ba-6f3c-44b4-a270-d3fcc3b15175 admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:16.761987 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc84f1ba-6f3c-44b4-a270-d3fcc3b15175 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:16.762262 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc84f1ba-6f3c-44b4-a270-d3fcc3b15175 admin admin] PATH_INFO: `/v3/projects/031ebc86f61d41018bec3c8abf53b26e/users/1f864c06f7ef46fd9211899da7cec39a/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:16.763251 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e680394-ced3-43b0-b7e3-5be7b39a1455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:16.764105 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e680394-ced3-43b0-b7e3-5be7b39a1455 admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:16.764519 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e680394-ced3-43b0-b7e3-5be7b39a1455 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:16.764851 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e680394-ced3-43b0-b7e3-5be7b39a1455 admin admin] PATH_INFO: `/v3/projects/e6c11c510dff44639ab5b023a6524347/users/2f772c8d5392431db3046ce17acbafdc/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:16.766453 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc84f1ba-6f3c-44b4-a270-d3fcc3b15175 admin admin] RBAC: Authorizing `identity:create_grant(project_id=031ebc86f61d41018bec3c8abf53b26e, user_id=1f864c06f7ef46fd9211899da7cec39a, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:16.770097 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc84f1ba-6f3c-44b4-a270-d3fcc3b15175 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:16.787766 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e680394-ced3-43b0-b7e3-5be7b39a1455 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6c11c510dff44639ab5b023a6524347, user_id=2f772c8d5392431db3046ce17acbafdc, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:16.795074 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e680394-ced3-43b0-b7e3-5be7b39a1455 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:16.805859 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 825/1630] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:13:16 2022] PUT /identity/v3/projects/031ebc86f61d41018bec3c8abf53b26e/users/1f864c06f7ef46fd9211899da7cec39a/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:16.806642 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc84f1ba-6f3c-44b4-a270-d3fcc3b15175 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:16.833355 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 806/1631] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:13:16 2022] PUT /identity/v3/projects/e6c11c510dff44639ab5b023a6524347/users/2f772c8d5392431db3046ce17acbafdc/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:16.834321 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e680394-ced3-43b0-b7e3-5be7b39a1455 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:16.899367 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc84f1ba-6f3c-44b4-a270-d3fcc3b15175 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:16.949099 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:16.950160 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e680394-ced3-43b0-b7e3-5be7b39a1455 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:16.956284 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e090827f-e20c-45eb-a210-0e2b9dab51a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:16.957178 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e090827f-e20c-45eb-a210-0e2b9dab51a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:16.957651 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e090827f-e20c-45eb-a210-0e2b9dab51a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:16.958076 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e090827f-e20c-45eb-a210-0e2b9dab51a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:16.958893 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e090827f-e20c-45eb-a210-0e2b9dab51a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:16.960922 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e090827f-e20c-45eb-a210-0e2b9dab51a8 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:16.967840 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:16.978561 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 807/1632] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:13:16 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:13:16.981184 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b9c77-69bc-4821-9ea3-6f06dca1fe5a None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:16.981641 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b9c77-69bc-4821-9ea3-6f06dca1fe5a None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:16.982098 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b9c77-69bc-4821-9ea3-6f06dca1fe5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:16.990524 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df038e1-2865-4a22-b5f9-3350c40957a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:16.992238 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df038e1-2865-4a22-b5f9-3350c40957a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:17.003840 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df038e1-2865-4a22-b5f9-3350c40957a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:17.004797 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df038e1-2865-4a22-b5f9-3350c40957a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:17.005924 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df038e1-2865-4a22-b5f9-3350c40957a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:17.010428 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df038e1-2865-4a22-b5f9-3350c40957a2 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:17.015836 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-df7b9c77-69bc-4821-9ea3-6f06dca1fe5a None None] MFA Rules not processed for user `1f864c06f7ef46fd9211899da7cec39a`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:17.017646 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-df7b9c77-69bc-4821-9ea3-6f06dca1fe5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:17.028270 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 826/1633] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:13:16 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:13:17.029178 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df038e1-2865-4a22-b5f9-3350c40957a2 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:17.048561 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df038e1-2865-4a22-b5f9-3350c40957a2 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:17.057457 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca552e6-20bb-44ed-8306-0583d3d3ab94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:17.058578 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca552e6-20bb-44ed-8306-0583d3d3ab94 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:17.059931 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca552e6-20bb-44ed-8306-0583d3d3ab94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:17.060535 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca552e6-20bb-44ed-8306-0583d3d3ab94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:17.061440 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca552e6-20bb-44ed-8306-0583d3d3ab94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:17.063869 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca552e6-20bb-44ed-8306-0583d3d3ab94 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:17.088256 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 827/1634] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:13:17 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:13:17.089264 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca552e6-20bb-44ed-8306-0583d3d3ab94 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:17.113964 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca552e6-20bb-44ed-8306-0583d3d3ab94 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:17.114326 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 808/1635] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:16 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:17.115393 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7b9c77-69bc-4821-9ea3-6f06dca1fe5a None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:17.127379 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7b9c77-69bc-4821-9ea3-6f06dca1fe5a None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:17.127747 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090a4a69-a36b-44ed-a843-77a875c0c006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:17.130566 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a4a69-a36b-44ed-a843-77a875c0c006 admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:17.131839 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a4a69-a36b-44ed-a843-77a875c0c006 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:17.132539 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a4a69-a36b-44ed-a843-77a875c0c006 admin admin] PATH_INFO: `/v3/projects/6cba181189ba42f097c59ec57be303fa/users/66492b2909294b0cafc1261b185d35c6/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:17.134112 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c93aedb-485d-42f6-8df7-22aaa2aa45e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:17.135342 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c93aedb-485d-42f6-8df7-22aaa2aa45e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:17.135816 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c93aedb-485d-42f6-8df7-22aaa2aa45e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:17.136307 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c93aedb-485d-42f6-8df7-22aaa2aa45e3 admin admin] PATH_INFO: `/v3/projects/1a5ba109e4ed45cea2cde2ae3b3dff13/users/ad899bf9479d4270ba8c9d35258282a3/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:17.141510 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c93aedb-485d-42f6-8df7-22aaa2aa45e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a5ba109e4ed45cea2cde2ae3b3dff13, user_id=ad899bf9479d4270ba8c9d35258282a3, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:17.142997 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c93aedb-485d-42f6-8df7-22aaa2aa45e3 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:17.178206 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090a4a69-a36b-44ed-a843-77a875c0c006 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cba181189ba42f097c59ec57be303fa, user_id=66492b2909294b0cafc1261b185d35c6, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:17.185422 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090a4a69-a36b-44ed-a843-77a875c0c006 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:17.187193 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 809/1636] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:13:17 2022] PUT /identity/v3/projects/1a5ba109e4ed45cea2cde2ae3b3dff13/users/ad899bf9479d4270ba8c9d35258282a3/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:17.188414 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c93aedb-485d-42f6-8df7-22aaa2aa45e3 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:17.218931 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 828/1637] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:13:17 2022] PUT /identity/v3/projects/6cba181189ba42f097c59ec57be303fa/users/66492b2909294b0cafc1261b185d35c6/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:17.218931 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090a4a69-a36b-44ed-a843-77a875c0c006 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:17.301292 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c93aedb-485d-42f6-8df7-22aaa2aa45e3 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:17.390378 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090a4a69-a36b-44ed-a843-77a875c0c006 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:17.399841 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f887a2-6915-4c83-8ac7-f9b39e6d1184 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:17.402903 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f887a2-6915-4c83-8ac7-f9b39e6d1184 service cinder] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:17.403385 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f887a2-6915-4c83-8ac7-f9b39e6d1184 service cinder] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:17.403820 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f887a2-6915-4c83-8ac7-f9b39e6d1184 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:17.430210 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aad9b5b-8890-463d-ae0f-9834a4f352c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:17.431143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad9b5b-8890-463d-ae0f-9834a4f352c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:17.431489 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad9b5b-8890-463d-ae0f-9834a4f352c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:17.431832 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad9b5b-8890-463d-ae0f-9834a4f352c0 admin admin] PATH_INFO: `/v3/projects/e6c11c510dff44639ab5b023a6524347/users/2f772c8d5392431db3046ce17acbafdc/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:17.434176 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aad9b5b-8890-463d-ae0f-9834a4f352c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6c11c510dff44639ab5b023a6524347, user_id=2f772c8d5392431db3046ce17acbafdc, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:17.437529 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aad9b5b-8890-463d-ae0f-9834a4f352c0 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:17.456615 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 810/1638] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:13:17 2022] PUT /identity/v3/projects/e6c11c510dff44639ab5b023a6524347/users/2f772c8d5392431db3046ce17acbafdc/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:17.465593 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d3d032-4875-4d9c-bde1-be94080e3821 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:17.465593 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d3d032-4875-4d9c-bde1-be94080e3821 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:17.465593 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d3d032-4875-4d9c-bde1-be94080e3821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:17.502134 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f887a2-6915-4c83-8ac7-f9b39e6d1184 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:17.504263 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f887a2-6915-4c83-8ac7-f9b39e6d1184 service cinder] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:17.507950 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:17.512617 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-79d3d032-4875-4d9c-bde1-be94080e3821 None None] MFA Rules not processed for user `ad899bf9479d4270ba8c9d35258282a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:17.514213 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-79d3d032-4875-4d9c-bde1-be94080e3821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:17.527646 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 829/1639] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:13:17 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:13:17.528866 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f887a2-6915-4c83-8ac7-f9b39e6d1184 service cinder] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:17.549817 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f887a2-6915-4c83-8ac7-f9b39e6d1184 service cinder] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:17.623672 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:17.666536 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1352cb-fe00-4303-9411-c8b90ddd3393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:17.667721 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1352cb-fe00-4303-9411-c8b90ddd3393 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:17.668217 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1352cb-fe00-4303-9411-c8b90ddd3393 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:17.668773 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1352cb-fe00-4303-9411-c8b90ddd3393 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:17.670327 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 811/1640] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:17 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:17.670727 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1352cb-fe00-4303-9411-c8b90ddd3393 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:17.671610 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e30d8-f465-4335-a9b7-109157357406 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:17.671960 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e30d8-f465-4335-a9b7-109157357406 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:17.672287 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e30d8-f465-4335-a9b7-109157357406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:17.675131 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1352cb-fe00-4303-9411-c8b90ddd3393 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:17.702244 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 830/1641] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:13:17 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:13:17.708429 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1352cb-fe00-4303-9411-c8b90ddd3393 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:17.742526 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-b66e30d8-f465-4335-a9b7-109157357406 None None] MFA Rules not processed for user `2f772c8d5392431db3046ce17acbafdc`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:17.744211 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1352cb-fe00-4303-9411-c8b90ddd3393 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:17.745825 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-b66e30d8-f465-4335-a9b7-109157357406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:17.756760 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8073c4a0-7b66-449a-84ed-c0f52f3092fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:17.757165 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8073c4a0-7b66-449a-84ed-c0f52f3092fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:17.759511 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8073c4a0-7b66-449a-84ed-c0f52f3092fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:17.759511 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8073c4a0-7b66-449a-84ed-c0f52f3092fe admin admin] PATH_INFO: `/v3/users/ad38d50e318441698ff2820858e8d499` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:17.761432 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8073c4a0-7b66-449a-84ed-c0f52f3092fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad38d50e318441698ff2820858e8d499)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:17.764091 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8073c4a0-7b66-449a-84ed-c0f52f3092fe admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:17.876374 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-8073c4a0-7b66-449a-84ed-c0f52f3092fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad38d50e318441698ff2820858e8d499'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:17.910238 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-8073c4a0-7b66-449a-84ed-c0f52f3092fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad38d50e318441698ff2820858e8d499'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:17.931471 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-8073c4a0-7b66-449a-84ed-c0f52f3092fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad38d50e318441698ff2820858e8d499'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:17.931676 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 812/1642] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:17 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:17.931676 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66e30d8-f465-4335-a9b7-109157357406 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:17.944221 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-8073c4a0-7b66-449a-84ed-c0f52f3092fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad38d50e318441698ff2820858e8d499'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:17.945602 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-8073c4a0-7b66-449a-84ed-c0f52f3092fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad38d50e318441698ff2820858e8d499'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:17.959058 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 831/1643] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:13:17 2022] DELETE /identity/v3/users/ad38d50e318441698ff2820858e8d499 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:17.966231 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8073c4a0-7b66-449a-84ed-c0f52f3092fe admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:17.969491 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-8073c4a0-7b66-449a-84ed-c0f52f3092fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:18.056949 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66e30d8-f465-4335-a9b7-109157357406 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:18.145146 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8073c4a0-7b66-449a-84ed-c0f52f3092fe admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:18.158881 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:18.161166 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e2da4-cfc7-4e7d-88e4-1261599e0c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:18.161959 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e2da4-cfc7-4e7d-88e4-1261599e0c16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:18.162212 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e2da4-cfc7-4e7d-88e4-1261599e0c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:18.162626 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e2da4-cfc7-4e7d-88e4-1261599e0c16 admin admin] PATH_INFO: `/v3/projects/6cba181189ba42f097c59ec57be303fa/users/66492b2909294b0cafc1261b185d35c6/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:18.166228 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1e2da4-cfc7-4e7d-88e4-1261599e0c16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cba181189ba42f097c59ec57be303fa, user_id=66492b2909294b0cafc1261b185d35c6, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:18.173985 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1e2da4-cfc7-4e7d-88e4-1261599e0c16 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:18.177664 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc855014-dd8f-4af7-9204-d7476649d368 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:18.178294 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc855014-dd8f-4af7-9204-d7476649d368 service cinder] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:18.178568 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc855014-dd8f-4af7-9204-d7476649d368 service cinder] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:18.178832 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc855014-dd8f-4af7-9204-d7476649d368 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:18.181057 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-bc855014-dd8f-4af7-9204-d7476649d368 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:18.195552 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 832/1644] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:13:17 2022] PUT /identity/v3/projects/6cba181189ba42f097c59ec57be303fa/users/66492b2909294b0cafc1261b185d35c6/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:18.196506 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e2da4-cfc7-4e7d-88e4-1261599e0c16 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:18.198325 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-5b1e2da4-cfc7-4e7d-88e4-1261599e0c16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:18.315635 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:18.324107 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e2da4-cfc7-4e7d-88e4-1261599e0c16 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:18.337466 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6211edb5-0c93-44e4-83e8-415e0b4fb2ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:18.338585 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6211edb5-0c93-44e4-83e8-415e0b4fb2ce service cinder] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:18.338879 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6211edb5-0c93-44e4-83e8-415e0b4fb2ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:18.339153 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6211edb5-0c93-44e4-83e8-415e0b4fb2ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:18.340595 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc855014-dd8f-4af7-9204-d7476649d368 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:18.342303 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-6211edb5-0c93-44e4-83e8-415e0b4fb2ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:18.342598 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc855014-dd8f-4af7-9204-d7476649d368 service cinder] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:18.363273 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 813/1645] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:13:17 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:13:18.363273 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a878b528-abe1-4abb-8ff6-f8254d526fe4 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:18.363273 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a878b528-abe1-4abb-8ff6-f8254d526fe4 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:18.363273 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a878b528-abe1-4abb-8ff6-f8254d526fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:18.386978 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-a878b528-abe1-4abb-8ff6-f8254d526fe4 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:18.388236 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-a878b528-abe1-4abb-8ff6-f8254d526fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:18.481342 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 814/1646] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:18.482904 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401cdb60-79eb-48c8-b82f-0e4fe52fdb46 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:18.483195 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401cdb60-79eb-48c8-b82f-0e4fe52fdb46 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:18.483543 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401cdb60-79eb-48c8-b82f-0e4fe52fdb46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:18.533352 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-401cdb60-79eb-48c8-b82f-0e4fe52fdb46 None None] MFA Rules not processed for user `66492b2909294b0cafc1261b185d35c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:18.537091 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-401cdb60-79eb-48c8-b82f-0e4fe52fdb46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:18.548106 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6211edb5-0c93-44e4-83e8-415e0b4fb2ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:18.549675 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6211edb5-0c93-44e4-83e8-415e0b4fb2ce service cinder] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:18.560749 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 833/1647] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:13:18 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:13:18.574389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6211edb5-0c93-44e4-83e8-415e0b4fb2ce service cinder] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:18.585345 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-6211edb5-0c93-44e4-83e8-415e0b4fb2ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:18.709170 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 815/1648] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:18 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:18.710251 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401cdb60-79eb-48c8-b82f-0e4fe52fdb46 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:18.712269 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-401cdb60-79eb-48c8-b82f-0e4fe52fdb46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:18.733242 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401cdb60-79eb-48c8-b82f-0e4fe52fdb46 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:18.741244 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593b0f01-2d8f-422e-a3da-27ee7584a2cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:18.742004 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593b0f01-2d8f-422e-a3da-27ee7584a2cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:18.742286 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593b0f01-2d8f-422e-a3da-27ee7584a2cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:18.742698 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593b0f01-2d8f-422e-a3da-27ee7584a2cd admin admin] PATH_INFO: `/v3/users/86f94747a92b4c3a8bbf2aad7d912758` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:18.744262 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593b0f01-2d8f-422e-a3da-27ee7584a2cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=86f94747a92b4c3a8bbf2aad7d912758)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:18.745508 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: WARNING py.warnings [None req-593b0f01-2d8f-422e-a3da-27ee7584a2cd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 01:13:18.745508 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: warnings.warn(msg) Sep 08 01:13:18.745508 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: Sep 08 01:13:18.748905 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593b0f01-2d8f-422e-a3da-27ee7584a2cd admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:18.758989 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6211edb5-0c93-44e4-83e8-415e0b4fb2ce service cinder] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:18.765881 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:18.768174 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ebf7ed-4afa-4577-a883-c06d40813c5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:18.770217 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ebf7ed-4afa-4577-a883-c06d40813c5b service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:18.770502 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ebf7ed-4afa-4577-a883-c06d40813c5b service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:18.770784 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ebf7ed-4afa-4577-a883-c06d40813c5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:18.796373 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ebf7ed-4afa-4577-a883-c06d40813c5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:18.798859 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ebf7ed-4afa-4577-a883-c06d40813c5b service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:18.830977 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 834/1649] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:13:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:13:18.832849 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ebf7ed-4afa-4577-a883-c06d40813c5b service neutron] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:18.839422 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-08ebf7ed-4afa-4577-a883-c06d40813c5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:18.866498 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ebf7ed-4afa-4577-a883-c06d40813c5b service neutron] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:18.874437 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-593b0f01-2d8f-422e-a3da-27ee7584a2cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86f94747a92b4c3a8bbf2aad7d912758'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:18.877641 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90961550-88d0-4a61-970f-b5eee8e59ee2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:18.878275 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90961550-88d0-4a61-970f-b5eee8e59ee2 service cinder] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:18.878556 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90961550-88d0-4a61-970f-b5eee8e59ee2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:18.878830 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90961550-88d0-4a61-970f-b5eee8e59ee2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:18.905265 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90961550-88d0-4a61-970f-b5eee8e59ee2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:18.906943 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90961550-88d0-4a61-970f-b5eee8e59ee2 service cinder] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:18.922904 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-593b0f01-2d8f-422e-a3da-27ee7584a2cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86f94747a92b4c3a8bbf2aad7d912758'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:18.940624 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 835/1650] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:13:18 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:13:18.944753 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-593b0f01-2d8f-422e-a3da-27ee7584a2cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86f94747a92b4c3a8bbf2aad7d912758'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:18.945753 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-593b0f01-2d8f-422e-a3da-27ee7584a2cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86f94747a92b4c3a8bbf2aad7d912758'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:18.962776 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-593b0f01-2d8f-422e-a3da-27ee7584a2cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86f94747a92b4c3a8bbf2aad7d912758'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:18.979632 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 816/1651] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:13:18 2022] DELETE /identity/v3/users/86f94747a92b4c3a8bbf2aad7d912758 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:18.983692 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e22165-dcf5-4ba3-b9f6-fec0908379c6 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:18.984218 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e22165-dcf5-4ba3-b9f6-fec0908379c6 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:18.984544 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e22165-dcf5-4ba3-b9f6-fec0908379c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:19.020924 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-86e22165-dcf5-4ba3-b9f6-fec0908379c6 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:19.022670 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-86e22165-dcf5-4ba3-b9f6-fec0908379c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:19.138689 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593b0f01-2d8f-422e-a3da-27ee7584a2cd admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:19.140009 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-593b0f01-2d8f-422e-a3da-27ee7584a2cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:19.185292 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 836/1652] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:13:19.353044 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593b0f01-2d8f-422e-a3da-27ee7584a2cd admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:19.360453 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b95de36-5b9d-46ed-b090-5af43d2c7a92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:19.361245 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b95de36-5b9d-46ed-b090-5af43d2c7a92 service cinder] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:19.361538 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b95de36-5b9d-46ed-b090-5af43d2c7a92 service cinder] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:19.361820 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b95de36-5b9d-46ed-b090-5af43d2c7a92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:19.365568 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-6b95de36-5b9d-46ed-b090-5af43d2c7a92 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:19.367692 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e22165-dcf5-4ba3-b9f6-fec0908379c6 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:19.369614 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-86e22165-dcf5-4ba3-b9f6-fec0908379c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:19.508055 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:19.524363 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b95de36-5b9d-46ed-b090-5af43d2c7a92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:19.525869 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b95de36-5b9d-46ed-b090-5af43d2c7a92 service cinder] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:19.542505 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 817/1653] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:13:19 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 01:13:19.578254 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e22165-dcf5-4ba3-b9f6-fec0908379c6 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:19.584431 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99762ff-9d4d-417d-b4fa-2f5a2c86b75b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:19.585185 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99762ff-9d4d-417d-b4fa-2f5a2c86b75b service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:19.585361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99762ff-9d4d-417d-b4fa-2f5a2c86b75b service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:19.585642 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99762ff-9d4d-417d-b4fa-2f5a2c86b75b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:19.608418 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99762ff-9d4d-417d-b4fa-2f5a2c86b75b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:19.610059 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99762ff-9d4d-417d-b4fa-2f5a2c86b75b service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:19.654578 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 837/1654] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:13:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 01:13:19.683219 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b95de36-5b9d-46ed-b090-5af43d2c7a92 service cinder] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:19.684908 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-6b95de36-5b9d-46ed-b090-5af43d2c7a92 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:19.711725 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b95de36-5b9d-46ed-b090-5af43d2c7a92 service cinder] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:19.741739 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b704e381-bd1a-418f-b1e2-74a7d8186c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:19.742634 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b704e381-bd1a-418f-b1e2-74a7d8186c2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:19.742990 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b704e381-bd1a-418f-b1e2-74a7d8186c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:19.743323 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b704e381-bd1a-418f-b1e2-74a7d8186c2b admin admin] PATH_INFO: `/v3/users/11b818b09de243da96959a412ee92075` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:19.745191 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b704e381-bd1a-418f-b1e2-74a7d8186c2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=11b818b09de243da96959a412ee92075)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:19.751642 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b704e381-bd1a-418f-b1e2-74a7d8186c2b admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:19.838051 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:19.872849 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-b704e381-bd1a-418f-b1e2-74a7d8186c2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11b818b09de243da96959a412ee92075'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:19.918790 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-b704e381-bd1a-418f-b1e2-74a7d8186c2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11b818b09de243da96959a412ee92075'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:19.947917 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-b704e381-bd1a-418f-b1e2-74a7d8186c2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11b818b09de243da96959a412ee92075'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:19.949708 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-b704e381-bd1a-418f-b1e2-74a7d8186c2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11b818b09de243da96959a412ee92075'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:19.971214 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-b704e381-bd1a-418f-b1e2-74a7d8186c2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11b818b09de243da96959a412ee92075'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:19.980323 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 818/1655] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:13:19 2022] DELETE /identity/v3/users/11b818b09de243da96959a412ee92075 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:19.984677 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301e269-0455-474e-8fe3-2ed4013f9e3c None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:19.985200 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301e269-0455-474e-8fe3-2ed4013f9e3c None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:19.985471 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301e269-0455-474e-8fe3-2ed4013f9e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:20.021216 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-c301e269-0455-474e-8fe3-2ed4013f9e3c None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:20.022524 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-c301e269-0455-474e-8fe3-2ed4013f9e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:20.173865 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 838/1656] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:20.224208 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b704e381-bd1a-418f-b1e2-74a7d8186c2b admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:20.225992 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-b704e381-bd1a-418f-b1e2-74a7d8186c2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:20.358271 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b704e381-bd1a-418f-b1e2-74a7d8186c2b admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:20.364154 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df269e-35cd-4691-bfab-ab42ff0d642b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:20.365705 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df269e-35cd-4691-bfab-ab42ff0d642b service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:20.366173 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df269e-35cd-4691-bfab-ab42ff0d642b service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:20.366545 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df269e-35cd-4691-bfab-ab42ff0d642b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:20.380787 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68df269e-35cd-4691-bfab-ab42ff0d642b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:20.382354 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68df269e-35cd-4691-bfab-ab42ff0d642b service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:20.395146 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 819/1657] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:13:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:13:21.354040 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c301e269-0455-474e-8fe3-2ed4013f9e3c None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:21.355590 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-c301e269-0455-474e-8fe3-2ed4013f9e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:21.380732 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c301e269-0455-474e-8fe3-2ed4013f9e3c None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:21.396502 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5a734a-5cd8-4907-a5bb-47163a443d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:21.397159 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a734a-5cd8-4907-a5bb-47163a443d95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:21.397443 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a734a-5cd8-4907-a5bb-47163a443d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:21.397725 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a734a-5cd8-4907-a5bb-47163a443d95 admin admin] PATH_INFO: `/v3/projects/18b694774b5b4608a768e77415dfcd02` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:21.398892 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5a734a-5cd8-4907-a5bb-47163a443d95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18b694774b5b4608a768e77415dfcd02)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:21.399917 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: WARNING py.warnings [None req-9c5a734a-5cd8-4907-a5bb-47163a443d95 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 01:13:21.399917 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: warnings.warn(msg) Sep 08 01:13:21.399917 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: Sep 08 01:13:21.401213 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5a734a-5cd8-4907-a5bb-47163a443d95 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:21.548562 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-9c5a734a-5cd8-4907-a5bb-47163a443d95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18b694774b5b4608a768e77415dfcd02', 'request_id': 'req-9c5a734a-5cd8-4907-a5bb-47163a443d95'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:21.584925 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-9c5a734a-5cd8-4907-a5bb-47163a443d95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18b694774b5b4608a768e77415dfcd02', 'request_id': 'req-9c5a734a-5cd8-4907-a5bb-47163a443d95'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:21.592990 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-9c5a734a-5cd8-4907-a5bb-47163a443d95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18b694774b5b4608a768e77415dfcd02', 'request_id': 'req-9c5a734a-5cd8-4907-a5bb-47163a443d95'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:21.621567 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-9c5a734a-5cd8-4907-a5bb-47163a443d95 admin admin] The token cache is being invalidate because project 18b694774b5b4608a768e77415dfcd02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:13:21.621858 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-9c5a734a-5cd8-4907-a5bb-47163a443d95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:21.632667 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-9c5a734a-5cd8-4907-a5bb-47163a443d95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:21.635826 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 839/1658] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:13:21 2022] DELETE /identity/v3/projects/18b694774b5b4608a768e77415dfcd02 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:21.639938 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df269e-35cd-4691-bfab-ab42ff0d642b service neutron] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:21.656968 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df269e-35cd-4691-bfab-ab42ff0d642b service neutron] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:21.766105 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5a734a-5cd8-4907-a5bb-47163a443d95 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:21.767943 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-9c5a734a-5cd8-4907-a5bb-47163a443d95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:21.833493 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdac834-81a3-476e-99fe-0e1575464101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:21.834176 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdac834-81a3-476e-99fe-0e1575464101 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:21.834562 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdac834-81a3-476e-99fe-0e1575464101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:21.834844 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdac834-81a3-476e-99fe-0e1575464101 admin admin] PATH_INFO: `/v3/users/2f772c8d5392431db3046ce17acbafdc` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:21.841400 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdac834-81a3-476e-99fe-0e1575464101 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f772c8d5392431db3046ce17acbafdc)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:21.844034 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdac834-81a3-476e-99fe-0e1575464101 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:21.944730 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:21.967436 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5a734a-5cd8-4907-a5bb-47163a443d95 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:21.979987 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69789263-2deb-42e9-8b4b-a002ec36b59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:21.984885 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69789263-2deb-42e9-8b4b-a002ec36b59d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:21.984885 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69789263-2deb-42e9-8b4b-a002ec36b59d admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:21.985181 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69789263-2deb-42e9-8b4b-a002ec36b59d admin admin] PATH_INFO: `/v3/projects/8eed1dc3232042248cd4adeaefdcc910` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:21.986637 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-dcdac834-81a3-476e-99fe-0e1575464101 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f772c8d5392431db3046ce17acbafdc'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:21.988854 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69789263-2deb-42e9-8b4b-a002ec36b59d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8eed1dc3232042248cd4adeaefdcc910)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:21.990839 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69789263-2deb-42e9-8b4b-a002ec36b59d admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:22.026690 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:22.042426 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-dcdac834-81a3-476e-99fe-0e1575464101 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f772c8d5392431db3046ce17acbafdc'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:22.080576 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-dcdac834-81a3-476e-99fe-0e1575464101 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f772c8d5392431db3046ce17acbafdc'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:22.082456 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-dcdac834-81a3-476e-99fe-0e1575464101 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f772c8d5392431db3046ce17acbafdc'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:22.103893 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-dcdac834-81a3-476e-99fe-0e1575464101 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f772c8d5392431db3046ce17acbafdc'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:22.118675 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 820/1659] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:13:21 2022] DELETE /identity/v3/users/2f772c8d5392431db3046ce17acbafdc => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:22.156379 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-69789263-2deb-42e9-8b4b-a002ec36b59d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8eed1dc3232042248cd4adeaefdcc910', 'request_id': 'req-69789263-2deb-42e9-8b4b-a002ec36b59d'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:22.177217 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-69789263-2deb-42e9-8b4b-a002ec36b59d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8eed1dc3232042248cd4adeaefdcc910', 'request_id': 'req-69789263-2deb-42e9-8b4b-a002ec36b59d'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:22.189511 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-69789263-2deb-42e9-8b4b-a002ec36b59d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8eed1dc3232042248cd4adeaefdcc910', 'request_id': 'req-69789263-2deb-42e9-8b4b-a002ec36b59d'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:22.228522 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-69789263-2deb-42e9-8b4b-a002ec36b59d admin admin] The token cache is being invalidate because project 8eed1dc3232042248cd4adeaefdcc910 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:13:22.228522 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-69789263-2deb-42e9-8b4b-a002ec36b59d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:22.229371 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-69789263-2deb-42e9-8b4b-a002ec36b59d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:22.239378 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 840/1660] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:13:21 2022] DELETE /identity/v3/projects/8eed1dc3232042248cd4adeaefdcc910 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:22.240931 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdac834-81a3-476e-99fe-0e1575464101 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:22.242363 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-dcdac834-81a3-476e-99fe-0e1575464101 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:22.402553 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdac834-81a3-476e-99fe-0e1575464101 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:22.526180 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f01190-8306-4ea4-a09c-2385dad06f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:22.528067 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f01190-8306-4ea4-a09c-2385dad06f56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:22.528799 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f01190-8306-4ea4-a09c-2385dad06f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:22.530492 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f01190-8306-4ea4-a09c-2385dad06f56 admin admin] PATH_INFO: `/v3/users/ad899bf9479d4270ba8c9d35258282a3` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:22.533870 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f01190-8306-4ea4-a09c-2385dad06f56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad899bf9479d4270ba8c9d35258282a3)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:22.536406 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f01190-8306-4ea4-a09c-2385dad06f56 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:22.648905 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-01f01190-8306-4ea4-a09c-2385dad06f56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad899bf9479d4270ba8c9d35258282a3'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:22.683353 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-01f01190-8306-4ea4-a09c-2385dad06f56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad899bf9479d4270ba8c9d35258282a3'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:22.696792 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-01f01190-8306-4ea4-a09c-2385dad06f56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad899bf9479d4270ba8c9d35258282a3'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:22.697660 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-01f01190-8306-4ea4-a09c-2385dad06f56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad899bf9479d4270ba8c9d35258282a3'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:22.711449 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-01f01190-8306-4ea4-a09c-2385dad06f56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad899bf9479d4270ba8c9d35258282a3'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:22.720879 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 821/1661] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:13:22 2022] DELETE /identity/v3/users/ad899bf9479d4270ba8c9d35258282a3 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:13:23.035532 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69789263-2deb-42e9-8b4b-a002ec36b59d admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:23.039053 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-69789263-2deb-42e9-8b4b-a002ec36b59d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:23.070057 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69789263-2deb-42e9-8b4b-a002ec36b59d admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:23.254546 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f83937-e665-4f1e-a21f-8dac06b18cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:23.257246 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f83937-e665-4f1e-a21f-8dac06b18cfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:23.257637 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f83937-e665-4f1e-a21f-8dac06b18cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:23.257807 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f83937-e665-4f1e-a21f-8dac06b18cfe admin admin] PATH_INFO: `/v3/projects/f33f4d95a2ba4eb693655b23f63fbe2e` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:23.260382 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f83937-e665-4f1e-a21f-8dac06b18cfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=f33f4d95a2ba4eb693655b23f63fbe2e)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:23.270024 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f83937-e665-4f1e-a21f-8dac06b18cfe admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:23.396746 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-38f83937-e665-4f1e-a21f-8dac06b18cfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f33f4d95a2ba4eb693655b23f63fbe2e', 'request_id': 'req-38f83937-e665-4f1e-a21f-8dac06b18cfe'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:23.407298 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-38f83937-e665-4f1e-a21f-8dac06b18cfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f33f4d95a2ba4eb693655b23f63fbe2e', 'request_id': 'req-38f83937-e665-4f1e-a21f-8dac06b18cfe'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:23.427712 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-38f83937-e665-4f1e-a21f-8dac06b18cfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f33f4d95a2ba4eb693655b23f63fbe2e', 'request_id': 'req-38f83937-e665-4f1e-a21f-8dac06b18cfe'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:23.446814 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-38f83937-e665-4f1e-a21f-8dac06b18cfe admin admin] The token cache is being invalidate because project f33f4d95a2ba4eb693655b23f63fbe2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:13:23.447113 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-38f83937-e665-4f1e-a21f-8dac06b18cfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:23.448321 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-38f83937-e665-4f1e-a21f-8dac06b18cfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:23.451736 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 841/1662] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:13:23 2022] DELETE /identity/v3/projects/f33f4d95a2ba4eb693655b23f63fbe2e => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:23.456306 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f01190-8306-4ea4-a09c-2385dad06f56 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:23.457817 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-01f01190-8306-4ea4-a09c-2385dad06f56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:23.466448 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:23.484276 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f01190-8306-4ea4-a09c-2385dad06f56 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:23.623681 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec33cae-7d3f-4a91-a0e9-434763dc5882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:23.624377 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec33cae-7d3f-4a91-a0e9-434763dc5882 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:23.624655 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec33cae-7d3f-4a91-a0e9-434763dc5882 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:23.631913 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec33cae-7d3f-4a91-a0e9-434763dc5882 admin admin] PATH_INFO: `/v3/users/1f864c06f7ef46fd9211899da7cec39a` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:23.631913 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec33cae-7d3f-4a91-a0e9-434763dc5882 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f864c06f7ef46fd9211899da7cec39a)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:23.632234 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec33cae-7d3f-4a91-a0e9-434763dc5882 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:23.757508 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-aec33cae-7d3f-4a91-a0e9-434763dc5882 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f864c06f7ef46fd9211899da7cec39a'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:23.777457 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-aec33cae-7d3f-4a91-a0e9-434763dc5882 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f864c06f7ef46fd9211899da7cec39a'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:23.786722 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-aec33cae-7d3f-4a91-a0e9-434763dc5882 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f864c06f7ef46fd9211899da7cec39a'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:23.787603 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-aec33cae-7d3f-4a91-a0e9-434763dc5882 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f864c06f7ef46fd9211899da7cec39a'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:23.809450 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-aec33cae-7d3f-4a91-a0e9-434763dc5882 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f864c06f7ef46fd9211899da7cec39a'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:23.832928 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 822/1663] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:13:23 2022] DELETE /identity/v3/users/1f864c06f7ef46fd9211899da7cec39a => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:23.877819 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f83937-e665-4f1e-a21f-8dac06b18cfe admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:23.878431 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-38f83937-e665-4f1e-a21f-8dac06b18cfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:23.899840 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f83937-e665-4f1e-a21f-8dac06b18cfe admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:23.926300 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:24.057119 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bb359c-2d5d-450d-a202-c2c78e023c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:24.058924 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb359c-2d5d-450d-a202-c2c78e023c3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:24.060787 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb359c-2d5d-450d-a202-c2c78e023c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:24.061378 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb359c-2d5d-450d-a202-c2c78e023c3a admin admin] PATH_INFO: `/v3/projects/e6c11c510dff44639ab5b023a6524347` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:24.066563 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bb359c-2d5d-450d-a202-c2c78e023c3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6c11c510dff44639ab5b023a6524347)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:24.068908 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bb359c-2d5d-450d-a202-c2c78e023c3a admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:24.157085 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec33cae-7d3f-4a91-a0e9-434763dc5882 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:24.159273 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-aec33cae-7d3f-4a91-a0e9-434763dc5882 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:24.177072 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-57bb359c-2d5d-450d-a202-c2c78e023c3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6c11c510dff44639ab5b023a6524347', 'request_id': 'req-57bb359c-2d5d-450d-a202-c2c78e023c3a'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:24.185249 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-57bb359c-2d5d-450d-a202-c2c78e023c3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6c11c510dff44639ab5b023a6524347', 'request_id': 'req-57bb359c-2d5d-450d-a202-c2c78e023c3a'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:24.190340 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec33cae-7d3f-4a91-a0e9-434763dc5882 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:24.197556 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-57bb359c-2d5d-450d-a202-c2c78e023c3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6c11c510dff44639ab5b023a6524347', 'request_id': 'req-57bb359c-2d5d-450d-a202-c2c78e023c3a'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:24.219022 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-57bb359c-2d5d-450d-a202-c2c78e023c3a admin admin] The token cache is being invalidate because project e6c11c510dff44639ab5b023a6524347 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:13:24.219584 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-57bb359c-2d5d-450d-a202-c2c78e023c3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:24.222048 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-57bb359c-2d5d-450d-a202-c2c78e023c3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:24.231315 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 842/1664] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:13:23 2022] DELETE /identity/v3/projects/e6c11c510dff44639ab5b023a6524347 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:24.232926 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bda28c-388b-4fed-b4e1-9749c7a072f2 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:24.233580 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bda28c-388b-4fed-b4e1-9749c7a072f2 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:24.235663 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bda28c-388b-4fed-b4e1-9749c7a072f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:24.294510 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-45bda28c-388b-4fed-b4e1-9749c7a072f2 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:24.295753 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-45bda28c-388b-4fed-b4e1-9749c7a072f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:24.407379 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bccb15-bbe1-424b-80cb-567534e01321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:24.408521 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bccb15-bbe1-424b-80cb-567534e01321 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:24.408810 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bccb15-bbe1-424b-80cb-567534e01321 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:24.409086 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bccb15-bbe1-424b-80cb-567534e01321 admin admin] PATH_INFO: `/v3/projects/1a5ba109e4ed45cea2cde2ae3b3dff13` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:24.411075 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bccb15-bbe1-424b-80cb-567534e01321 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a5ba109e4ed45cea2cde2ae3b3dff13)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:24.414223 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bccb15-bbe1-424b-80cb-567534e01321 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:24.512189 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:24.542294 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 843/1665] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:24.555303 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bde3ee-92ae-4b98-ae60-6099e753b15d None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:24.555592 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bde3ee-92ae-4b98-ae60-6099e753b15d None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:24.555871 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bde3ee-92ae-4b98-ae60-6099e753b15d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:24.583822 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-00bccb15-bbe1-424b-80cb-567534e01321 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a5ba109e4ed45cea2cde2ae3b3dff13', 'request_id': 'req-00bccb15-bbe1-424b-80cb-567534e01321'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:24.600621 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-00bccb15-bbe1-424b-80cb-567534e01321 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a5ba109e4ed45cea2cde2ae3b3dff13', 'request_id': 'req-00bccb15-bbe1-424b-80cb-567534e01321'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:24.610669 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-d1bde3ee-92ae-4b98-ae60-6099e753b15d None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:24.612850 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-00bccb15-bbe1-424b-80cb-567534e01321 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a5ba109e4ed45cea2cde2ae3b3dff13', 'request_id': 'req-00bccb15-bbe1-424b-80cb-567534e01321'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:24.616996 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-d1bde3ee-92ae-4b98-ae60-6099e753b15d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:24.655481 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-00bccb15-bbe1-424b-80cb-567534e01321 admin admin] The token cache is being invalidate because project 1a5ba109e4ed45cea2cde2ae3b3dff13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81661) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:13:24.655777 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-00bccb15-bbe1-424b-80cb-567534e01321 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:24.658572 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-00bccb15-bbe1-424b-80cb-567534e01321 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:24.666807 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 823/1666] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:13:24 2022] DELETE /identity/v3/projects/1a5ba109e4ed45cea2cde2ae3b3dff13 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:24.670798 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27dda9-6d5e-4c79-853e-bc8f8e594097 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:24.671082 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27dda9-6d5e-4c79-853e-bc8f8e594097 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:24.671536 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27dda9-6d5e-4c79-853e-bc8f8e594097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:24.725424 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-ca27dda9-6d5e-4c79-853e-bc8f8e594097 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:24.732316 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-ca27dda9-6d5e-4c79-853e-bc8f8e594097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:24.748778 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:24.799588 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 844/1667] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:24.804224 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bde3ee-92ae-4b98-ae60-6099e753b15d None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:24.823523 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bde3ee-92ae-4b98-ae60-6099e753b15d None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:24.838189 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3731ef0b-ca0c-4666-b331-1b14cc8339b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:24.839115 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3731ef0b-ca0c-4666-b331-1b14cc8339b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:24.839398 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3731ef0b-ca0c-4666-b331-1b14cc8339b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:24.839674 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3731ef0b-ca0c-4666-b331-1b14cc8339b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:24.840439 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3731ef0b-ca0c-4666-b331-1b14cc8339b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:13:24.840748 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3731ef0b-ca0c-4666-b331-1b14cc8339b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:24.842406 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3731ef0b-ca0c-4666-b331-1b14cc8339b9 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:24.875961 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 845/1668] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:13:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:13:24.877522 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3731ef0b-ca0c-4666-b331-1b14cc8339b9 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:24.893653 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 824/1669] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:24.899908 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3731ef0b-ca0c-4666-b331-1b14cc8339b9 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:24.902636 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b6d70-24ee-4118-9870-8dfa96382dc7 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:24.903025 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b6d70-24ee-4118-9870-8dfa96382dc7 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:24.903304 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b6d70-24ee-4118-9870-8dfa96382dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:24.918512 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dad3a8-9b86-4072-8d72-08f9d7a5abe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:24.919605 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dad3a8-9b86-4072-8d72-08f9d7a5abe7 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:24.922902 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dad3a8-9b86-4072-8d72-08f9d7a5abe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:24.923776 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dad3a8-9b86-4072-8d72-08f9d7a5abe7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:24.927423 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dad3a8-9b86-4072-8d72-08f9d7a5abe7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:24.931602 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dad3a8-9b86-4072-8d72-08f9d7a5abe7 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:24.944571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-b97b6d70-24ee-4118-9870-8dfa96382dc7 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:24.949455 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-b97b6d70-24ee-4118-9870-8dfa96382dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:24.969183 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 825/1670] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:24.970900 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97b6d70-24ee-4118-9870-8dfa96382dc7 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:24.998133 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 846/1671] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:13:24 2022] POST /identity/v3/projects => generated 355 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:13:24.999115 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dad3a8-9b86-4072-8d72-08f9d7a5abe7 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:25.002565 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97b6d70-24ee-4118-9870-8dfa96382dc7 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:25.156671 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f5673a-e4b4-44ee-ba7b-bc1fe6dea50b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:25.160452 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f5673a-e4b4-44ee-ba7b-bc1fe6dea50b admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:25.160741 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f5673a-e4b4-44ee-ba7b-bc1fe6dea50b admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:25.161016 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f5673a-e4b4-44ee-ba7b-bc1fe6dea50b admin admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:25.161699 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f5673a-e4b4-44ee-ba7b-bc1fe6dea50b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:13:25.163595 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f5673a-e4b4-44ee-ba7b-bc1fe6dea50b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:25.165310 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f5673a-e4b4-44ee-ba7b-bc1fe6dea50b admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:25.177200 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dad3a8-9b86-4072-8d72-08f9d7a5abe7 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:25.190850 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca928e-c078-49e3-8746-0736657ba6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:25.191481 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca928e-c078-49e3-8746-0736657ba6ff admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:25.191762 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca928e-c078-49e3-8746-0736657ba6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:25.192034 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca928e-c078-49e3-8746-0736657ba6ff admin admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:25.193022 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afca928e-c078-49e3-8746-0736657ba6ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:25.197159 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afca928e-c078-49e3-8746-0736657ba6ff admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:25.202966 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 826/1672] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:13:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:13:25.203985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f5673a-e4b4-44ee-ba7b-bc1fe6dea50b admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:25.223578 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f5673a-e4b4-44ee-ba7b-bc1fe6dea50b admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:25.234713 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a941009e-624a-4393-9984-075e821903a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:25.236438 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941009e-624a-4393-9984-075e821903a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:25.236997 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941009e-624a-4393-9984-075e821903a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:25.238369 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941009e-624a-4393-9984-075e821903a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:25.239337 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a941009e-624a-4393-9984-075e821903a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:25.241401 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a941009e-624a-4393-9984-075e821903a0 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:25.245978 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 847/1673] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:13:24 2022] POST /identity/v3/users => generated 278 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:13:25.250983 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca928e-c078-49e3-8746-0736657ba6ff admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:25.274016 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca928e-c078-49e3-8746-0736657ba6ff admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:25.297321 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552f46a3-99db-447f-a439-63976c0afd8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:25.299706 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f46a3-99db-447f-a439-63976c0afd8f admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:25.300476 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f46a3-99db-447f-a439-63976c0afd8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:25.300476 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f46a3-99db-447f-a439-63976c0afd8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:25.301274 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552f46a3-99db-447f-a439-63976c0afd8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:25.308043 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 827/1674] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:13:25 2022] POST /identity/v3/projects => generated 373 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 01:13:25.309254 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a941009e-624a-4393-9984-075e821903a0 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:25.309903 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552f46a3-99db-447f-a439-63976c0afd8f admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:25.337242 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 848/1675] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:13:25.338432 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552f46a3-99db-447f-a439-63976c0afd8f admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:25.368406 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:25.370021 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552f46a3-99db-447f-a439-63976c0afd8f admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:25.490636 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a941009e-624a-4393-9984-075e821903a0 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:25.492082 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f869a-6e15-4da9-8c51-02520a8e7fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:25.493096 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f869a-6e15-4da9-8c51-02520a8e7fa4 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:25.493526 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f869a-6e15-4da9-8c51-02520a8e7fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:25.493946 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f869a-6e15-4da9-8c51-02520a8e7fa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:25.494749 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5f869a-6e15-4da9-8c51-02520a8e7fa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:25.496848 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5f869a-6e15-4da9-8c51-02520a8e7fa4 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:25.503861 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d12dc2-f555-434a-be50-49e5d681b60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:25.507355 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d12dc2-f555-434a-be50-49e5d681b60e admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:25.507652 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d12dc2-f555-434a-be50-49e5d681b60e admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:25.507924 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d12dc2-f555-434a-be50-49e5d681b60e admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:25.508893 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d12dc2-f555-434a-be50-49e5d681b60e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:25.511302 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d12dc2-f555-434a-be50-49e5d681b60e admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:25.520430 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 849/1676] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:13:25.521643 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f869a-6e15-4da9-8c51-02520a8e7fa4 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:25.543562 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f869a-6e15-4da9-8c51-02520a8e7fa4 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:25.567231 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 828/1677] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:13:25 2022] POST /identity/v3/users => generated 287 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:13:25.571196 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d12dc2-f555-434a-be50-49e5d681b60e admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:25.573738 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b0c77d-6252-4b96-b3d2-bfc9829fbdb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:25.574431 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b0c77d-6252-4b96-b3d2-bfc9829fbdb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:25.574712 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b0c77d-6252-4b96-b3d2-bfc9829fbdb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:25.574986 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b0c77d-6252-4b96-b3d2-bfc9829fbdb7 admin admin] PATH_INFO: `/v3/projects/861dc116bd6b420f942982c507e6fb14/users/a483fa35bddf4ceab6725166c2fc87d7/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:25.590357 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d12dc2-f555-434a-be50-49e5d681b60e admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:25.598422 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e816c4f-f990-41b9-b462-089d88c7a9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:25.599258 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e816c4f-f990-41b9-b462-089d88c7a9a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:25.600163 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e816c4f-f990-41b9-b462-089d88c7a9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:25.600488 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e816c4f-f990-41b9-b462-089d88c7a9a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:25.601146 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e816c4f-f990-41b9-b462-089d88c7a9a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:25.602778 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e816c4f-f990-41b9-b462-089d88c7a9a6 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:25.624949 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b0c77d-6252-4b96-b3d2-bfc9829fbdb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=861dc116bd6b420f942982c507e6fb14, user_id=a483fa35bddf4ceab6725166c2fc87d7, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:25.625364 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 829/1678] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:13:25.628504 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b0c77d-6252-4b96-b3d2-bfc9829fbdb7 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:25.641851 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e816c4f-f990-41b9-b462-089d88c7a9a6 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:25.666610 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 850/1679] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:13:25 2022] PUT /identity/v3/projects/861dc116bd6b420f942982c507e6fb14/users/a483fa35bddf4ceab6725166c2fc87d7/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:25.666831 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e816c4f-f990-41b9-b462-089d88c7a9a6 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:25.668328 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d69cdb-44f8-4061-808b-381d27cd31b0 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:25.668620 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d69cdb-44f8-4061-808b-381d27cd31b0 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:25.668892 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d69cdb-44f8-4061-808b-381d27cd31b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:25.681619 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:25.743409 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-67d69cdb-44f8-4061-808b-381d27cd31b0 None None] MFA Rules not processed for user `a483fa35bddf4ceab6725166c2fc87d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:25.744692 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-67d69cdb-44f8-4061-808b-381d27cd31b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:25.770222 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354a2cbc-2a9d-44b1-b60d-82e0f9469671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:25.771111 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a2cbc-2a9d-44b1-b60d-82e0f9469671 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:25.773926 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a2cbc-2a9d-44b1-b60d-82e0f9469671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:25.774400 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a2cbc-2a9d-44b1-b60d-82e0f9469671 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:25.775936 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354a2cbc-2a9d-44b1-b60d-82e0f9469671 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:25.780478 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354a2cbc-2a9d-44b1-b60d-82e0f9469671 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:25.818107 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 830/1680] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:13:25.821809 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354a2cbc-2a9d-44b1-b60d-82e0f9469671 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:25.860236 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354a2cbc-2a9d-44b1-b60d-82e0f9469671 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:25.880500 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2df87ad-5043-4317-8754-72812876d3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:25.881417 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2df87ad-5043-4317-8754-72812876d3c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:25.882365 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2df87ad-5043-4317-8754-72812876d3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:25.883725 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2df87ad-5043-4317-8754-72812876d3c5 admin admin] PATH_INFO: `/v3/projects/20824692a9a64ceeb7e81aad2470ff3b/users/81c978962290481b828282a516f089bc/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:25.894836 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 851/1681] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:25 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:13:25.895985 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d69cdb-44f8-4061-808b-381d27cd31b0 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:25.919221 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d69cdb-44f8-4061-808b-381d27cd31b0 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:25.928981 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2df87ad-5043-4317-8754-72812876d3c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20824692a9a64ceeb7e81aad2470ff3b, user_id=81c978962290481b828282a516f089bc, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:25.934282 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3914504-ad5b-4979-8594-b35c26b695f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:25.935170 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3914504-ad5b-4979-8594-b35c26b695f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:25.935401 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3914504-ad5b-4979-8594-b35c26b695f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:25.936045 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3914504-ad5b-4979-8594-b35c26b695f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:25.936488 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3914504-ad5b-4979-8594-b35c26b695f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:25.940168 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3914504-ad5b-4979-8594-b35c26b695f4 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:25.941113 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2df87ad-5043-4317-8754-72812876d3c5 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:25.969766 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 831/1682] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:13:25 2022] PUT /identity/v3/projects/20824692a9a64ceeb7e81aad2470ff3b/users/81c978962290481b828282a516f089bc/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:25.980574 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a785bb28-61c0-4e7b-bd1d-48cf83a7466d None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:25.980904 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a785bb28-61c0-4e7b-bd1d-48cf83a7466d None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:25.981097 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a785bb28-61c0-4e7b-bd1d-48cf83a7466d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:26.018546 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 852/1683] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:13:25 2022] POST /identity/v3/projects => generated 355 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 01:13:26.020347 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3914504-ad5b-4979-8594-b35c26b695f4 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:26.022561 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-b3914504-ad5b-4979-8594-b35c26b695f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:26.046160 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-a785bb28-61c0-4e7b-bd1d-48cf83a7466d None None] MFA Rules not processed for user `81c978962290481b828282a516f089bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:26.049205 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-a785bb28-61c0-4e7b-bd1d-48cf83a7466d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:26.120788 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3914504-ad5b-4979-8594-b35c26b695f4 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:26.129603 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47383ec1-f1d4-4823-815d-e923b4c97f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:26.131151 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47383ec1-f1d4-4823-815d-e923b4c97f4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:26.131238 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47383ec1-f1d4-4823-815d-e923b4c97f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:26.131513 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47383ec1-f1d4-4823-815d-e923b4c97f4f admin admin] PATH_INFO: `/v3/projects/031ebc86f61d41018bec3c8abf53b26e` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:26.153171 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47383ec1-f1d4-4823-815d-e923b4c97f4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=031ebc86f61d41018bec3c8abf53b26e)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:26.155284 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47383ec1-f1d4-4823-815d-e923b4c97f4f admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:26.234842 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:26.246491 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 832/1684] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:13:26.247646 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a785bb28-61c0-4e7b-bd1d-48cf83a7466d None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:26.263073 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a785bb28-61c0-4e7b-bd1d-48cf83a7466d None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:26.290774 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-47383ec1-f1d4-4823-815d-e923b4c97f4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '031ebc86f61d41018bec3c8abf53b26e', 'request_id': 'req-47383ec1-f1d4-4823-815d-e923b4c97f4f'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:26.321294 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-47383ec1-f1d4-4823-815d-e923b4c97f4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '031ebc86f61d41018bec3c8abf53b26e', 'request_id': 'req-47383ec1-f1d4-4823-815d-e923b4c97f4f'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:26.330092 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-47383ec1-f1d4-4823-815d-e923b4c97f4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '031ebc86f61d41018bec3c8abf53b26e', 'request_id': 'req-47383ec1-f1d4-4823-815d-e923b4c97f4f'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:26.345737 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:26.352396 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-47383ec1-f1d4-4823-815d-e923b4c97f4f admin admin] The token cache is being invalidate because project 031ebc86f61d41018bec3c8abf53b26e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:13:26.353136 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-47383ec1-f1d4-4823-815d-e923b4c97f4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:26.354508 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-47383ec1-f1d4-4823-815d-e923b4c97f4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:26.357103 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 853/1685] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:13:26 2022] DELETE /identity/v3/projects/031ebc86f61d41018bec3c8abf53b26e => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:26.363464 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47383ec1-f1d4-4823-815d-e923b4c97f4f admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:26.370412 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-47383ec1-f1d4-4823-815d-e923b4c97f4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:26.402936 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17875526-34b2-4045-ae3d-c05f947545d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:26.403603 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17875526-34b2-4045-ae3d-c05f947545d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:26.405290 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17875526-34b2-4045-ae3d-c05f947545d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:26.405572 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17875526-34b2-4045-ae3d-c05f947545d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:26.406229 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17875526-34b2-4045-ae3d-c05f947545d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:26.413300 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17875526-34b2-4045-ae3d-c05f947545d9 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:26.542415 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 833/1686] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:13:26 2022] POST /identity/v3/users => generated 284 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:13:26.543752 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558efc6-928e-44ff-8c16-3a912bfede8c None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:26.544035 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558efc6-928e-44ff-8c16-3a912bfede8c None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:26.552396 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558efc6-928e-44ff-8c16-3a912bfede8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:26.561910 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47383ec1-f1d4-4823-815d-e923b4c97f4f admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:26.578363 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d9ab6-ebd2-4d5d-9bc9-86e7123e48b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:26.581813 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d9ab6-ebd2-4d5d-9bc9-86e7123e48b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:26.584536 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d9ab6-ebd2-4d5d-9bc9-86e7123e48b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:26.584840 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d9ab6-ebd2-4d5d-9bc9-86e7123e48b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:26.586338 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32d9ab6-ebd2-4d5d-9bc9-86e7123e48b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:26.589355 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-e558efc6-928e-44ff-8c16-3a912bfede8c None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:26.591020 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-e558efc6-928e-44ff-8c16-3a912bfede8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:26.591950 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32d9ab6-ebd2-4d5d-9bc9-86e7123e48b7 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:26.609126 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 834/1687] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:26.613350 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e558efc6-928e-44ff-8c16-3a912bfede8c None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:26.614918 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-e558efc6-928e-44ff-8c16-3a912bfede8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:26.644211 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e558efc6-928e-44ff-8c16-3a912bfede8c None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:26.650908 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9577fbf1-03d6-4348-a12a-c5c31b9d5458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:26.651754 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577fbf1-03d6-4348-a12a-c5c31b9d5458 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:26.652039 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577fbf1-03d6-4348-a12a-c5c31b9d5458 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:26.652339 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577fbf1-03d6-4348-a12a-c5c31b9d5458 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:26.653140 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9577fbf1-03d6-4348-a12a-c5c31b9d5458 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:26.654869 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9577fbf1-03d6-4348-a12a-c5c31b9d5458 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:26.678407 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 835/1688] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:13:26.678603 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e2a223-b6a9-4d6a-8944-4bc367a17ecd None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:26.678831 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e2a223-b6a9-4d6a-8944-4bc367a17ecd None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:26.679322 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e2a223-b6a9-4d6a-8944-4bc367a17ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:26.694301 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 854/1689] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:13:26 2022] POST /identity/v3/projects => generated 373 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:13:26.705435 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d9ab6-ebd2-4d5d-9bc9-86e7123e48b7 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:26.720303 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-52e2a223-b6a9-4d6a-8944-4bc367a17ecd None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:26.722219 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-52e2a223-b6a9-4d6a-8944-4bc367a17ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:26.934889 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 836/1690] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:26.936436 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e2a223-b6a9-4d6a-8944-4bc367a17ecd None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:26.939892 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d9ab6-ebd2-4d5d-9bc9-86e7123e48b7 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:26.968021 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e2a223-b6a9-4d6a-8944-4bc367a17ecd None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:26.976332 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffba2f6-75d2-49d1-a6cd-a9ded98842ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:26.977433 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffba2f6-75d2-49d1-a6cd-a9ded98842ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:26.978688 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffba2f6-75d2-49d1-a6cd-a9ded98842ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:26.979949 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffba2f6-75d2-49d1-a6cd-a9ded98842ba admin admin] PATH_INFO: `/v3/projects/209cebd89073488caec6044bfc794fdd/users/5510182c76f1481a85bbc3dde966ce7d/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:26.984040 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0a43a6-3256-4ca7-a753-1bba9ecaf6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:26.984699 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a43a6-3256-4ca7-a753-1bba9ecaf6ab admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:26.984979 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a43a6-3256-4ca7-a753-1bba9ecaf6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:26.985250 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a43a6-3256-4ca7-a753-1bba9ecaf6ab admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:26.986155 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0a43a6-3256-4ca7-a753-1bba9ecaf6ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:26.992702 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0a43a6-3256-4ca7-a753-1bba9ecaf6ab admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:27.036908 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffba2f6-75d2-49d1-a6cd-a9ded98842ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=209cebd89073488caec6044bfc794fdd, user_id=5510182c76f1481a85bbc3dde966ce7d, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:27.041249 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 837/1691] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:13:26 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:13:27.042218 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0a43a6-3256-4ca7-a753-1bba9ecaf6ab admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:27.052512 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffba2f6-75d2-49d1-a6cd-a9ded98842ba admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:27.069982 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0a43a6-3256-4ca7-a753-1bba9ecaf6ab admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:27.070170 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 855/1692] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:13:26 2022] PUT /identity/v3/projects/209cebd89073488caec6044bfc794fdd/users/5510182c76f1481a85bbc3dde966ce7d/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:27.071001 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffba2f6-75d2-49d1-a6cd-a9ded98842ba admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:27.212502 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:27.238356 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388857f6-eb03-4be4-89ea-f9b728ee38a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:27.238995 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388857f6-eb03-4be4-89ea-f9b728ee38a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:27.239305 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388857f6-eb03-4be4-89ea-f9b728ee38a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:27.239577 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388857f6-eb03-4be4-89ea-f9b728ee38a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:27.240262 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388857f6-eb03-4be4-89ea-f9b728ee38a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:13:27.240568 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388857f6-eb03-4be4-89ea-f9b728ee38a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:27.243694 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffba2f6-75d2-49d1-a6cd-a9ded98842ba admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:27.245371 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388857f6-eb03-4be4-89ea-f9b728ee38a8 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:27.269460 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8dd1b3-9242-4886-a571-eadb26f70f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:27.270185 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8dd1b3-9242-4886-a571-eadb26f70f21 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:27.270474 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8dd1b3-9242-4886-a571-eadb26f70f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:27.270747 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8dd1b3-9242-4886-a571-eadb26f70f21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:27.271409 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8dd1b3-9242-4886-a571-eadb26f70f21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:27.275063 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8dd1b3-9242-4886-a571-eadb26f70f21 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:27.288459 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 838/1693] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:13:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:13:27.295061 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388857f6-eb03-4be4-89ea-f9b728ee38a8 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:27.309479 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 856/1694] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:13:27.309479 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8dd1b3-9242-4886-a571-eadb26f70f21 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:27.325797 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:27.331132 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388857f6-eb03-4be4-89ea-f9b728ee38a8 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:27.336636 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8dd1b3-9242-4886-a571-eadb26f70f21 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:27.346862 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e79534-9517-4534-901b-58d5b78460b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:27.347503 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e79534-9517-4534-901b-58d5b78460b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:27.347785 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e79534-9517-4534-901b-58d5b78460b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:27.348057 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e79534-9517-4534-901b-58d5b78460b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:27.349586 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e79534-9517-4534-901b-58d5b78460b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:27.351224 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e79534-9517-4534-901b-58d5b78460b8 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:27.359991 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3278b6e-fa99-4600-8726-69e8d5d3d6b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:27.361465 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3278b6e-fa99-4600-8726-69e8d5d3d6b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:27.361777 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3278b6e-fa99-4600-8726-69e8d5d3d6b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:27.362209 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3278b6e-fa99-4600-8726-69e8d5d3d6b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:27.362835 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3278b6e-fa99-4600-8726-69e8d5d3d6b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:27.369759 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3278b6e-fa99-4600-8726-69e8d5d3d6b6 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:27.394609 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 839/1695] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:13:27.396168 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e79534-9517-4534-901b-58d5b78460b8 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:27.418115 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e79534-9517-4534-901b-58d5b78460b8 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:27.431087 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c7e865-4536-42af-8c69-a88a5e91e653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:27.432153 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7e865-4536-42af-8c69-a88a5e91e653 admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:27.433170 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7e865-4536-42af-8c69-a88a5e91e653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:27.434888 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7e865-4536-42af-8c69-a88a5e91e653 admin admin] PATH_INFO: `/v3/projects/a31d05c114444ca7948f87e2a5d591a4/users/b16e336cbee349de97bbc0d8c6c73c2e/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:27.438253 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 857/1696] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:13:27 2022] POST /identity/v3/projects => generated 399 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:13:27.444819 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3278b6e-fa99-4600-8726-69e8d5d3d6b6 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:27.485894 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c7e865-4536-42af-8c69-a88a5e91e653 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a31d05c114444ca7948f87e2a5d591a4, user_id=b16e336cbee349de97bbc0d8c6c73c2e, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:27.496553 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c7e865-4536-42af-8c69-a88a5e91e653 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:27.537786 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 840/1697] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:13:27 2022] PUT /identity/v3/projects/a31d05c114444ca7948f87e2a5d591a4/users/b16e336cbee349de97bbc0d8c6c73c2e/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:27.539022 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c7e865-4536-42af-8c69-a88a5e91e653 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:27.641707 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3278b6e-fa99-4600-8726-69e8d5d3d6b6 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:27.707888 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c7e865-4536-42af-8c69-a88a5e91e653 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:27.722812 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18087279-638d-44c6-b22b-f44d699c53d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:27.723762 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18087279-638d-44c6-b22b-f44d699c53d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:27.724165 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18087279-638d-44c6-b22b-f44d699c53d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:27.724800 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18087279-638d-44c6-b22b-f44d699c53d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:27.725567 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18087279-638d-44c6-b22b-f44d699c53d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:27.730135 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18087279-638d-44c6-b22b-f44d699c53d6 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:27.779427 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 841/1698] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:13:27 2022] POST /identity/v3/users => generated 300 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:13:27.784315 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18087279-638d-44c6-b22b-f44d699c53d6 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:27.786405 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb073d6b-8bd0-4f26-a142-69f76cda992e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:27.787575 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb073d6b-8bd0-4f26-a142-69f76cda992e admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:27.791221 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb073d6b-8bd0-4f26-a142-69f76cda992e admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:27.791637 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb073d6b-8bd0-4f26-a142-69f76cda992e admin admin] PATH_INFO: `/v3/projects/209cebd89073488caec6044bfc794fdd/users/5510182c76f1481a85bbc3dde966ce7d/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:27.794459 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb073d6b-8bd0-4f26-a142-69f76cda992e admin admin] RBAC: Authorizing `identity:create_grant(project_id=209cebd89073488caec6044bfc794fdd, user_id=5510182c76f1481a85bbc3dde966ce7d, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:27.800540 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb073d6b-8bd0-4f26-a142-69f76cda992e admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:27.808307 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18087279-638d-44c6-b22b-f44d699c53d6 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:27.820073 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7066f63e-7cea-460b-84f4-66566aff1dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:27.820833 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066f63e-7cea-460b-84f4-66566aff1dee admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:27.821240 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066f63e-7cea-460b-84f4-66566aff1dee admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:27.822274 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066f63e-7cea-460b-84f4-66566aff1dee admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:27.823037 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7066f63e-7cea-460b-84f4-66566aff1dee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:27.828291 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7066f63e-7cea-460b-84f4-66566aff1dee admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:27.838367 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 858/1699] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:13:27 2022] PUT /identity/v3/projects/209cebd89073488caec6044bfc794fdd/users/5510182c76f1481a85bbc3dde966ce7d/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:27.839478 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb073d6b-8bd0-4f26-a142-69f76cda992e admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:27.864634 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 842/1700] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:13:27.866064 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f52551-fa51-4e87-af8b-2f407c092600 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:27.866349 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f52551-fa51-4e87-af8b-2f407c092600 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:27.870383 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f52551-fa51-4e87-af8b-2f407c092600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:27.936800 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-c4f52551-fa51-4e87-af8b-2f407c092600 None None] MFA Rules not processed for user `5510182c76f1481a85bbc3dde966ce7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:27.938139 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-c4f52551-fa51-4e87-af8b-2f407c092600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:27.988441 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb073d6b-8bd0-4f26-a142-69f76cda992e admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:28.002275 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c042de72-c72e-4b89-8123-62b0906ae1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:28.006413 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042de72-c72e-4b89-8123-62b0906ae1b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:28.006704 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042de72-c72e-4b89-8123-62b0906ae1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:28.006981 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042de72-c72e-4b89-8123-62b0906ae1b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:28.007660 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c042de72-c72e-4b89-8123-62b0906ae1b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:28.009599 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c042de72-c72e-4b89-8123-62b0906ae1b8 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:28.035097 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 859/1701] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:13:28.042618 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c042de72-c72e-4b89-8123-62b0906ae1b8 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:28.065169 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c042de72-c72e-4b89-8123-62b0906ae1b8 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:28.078958 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb79d2d-1010-4d8f-8465-ea59401482a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:28.080210 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb79d2d-1010-4d8f-8465-ea59401482a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:28.080487 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb79d2d-1010-4d8f-8465-ea59401482a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:28.080756 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb79d2d-1010-4d8f-8465-ea59401482a9 admin admin] PATH_INFO: `/v3/projects/a31d05c114444ca7948f87e2a5d591a4/users/b16e336cbee349de97bbc0d8c6c73c2e/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:28.088239 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb79d2d-1010-4d8f-8465-ea59401482a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a31d05c114444ca7948f87e2a5d591a4, user_id=b16e336cbee349de97bbc0d8c6c73c2e, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:28.090951 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb79d2d-1010-4d8f-8465-ea59401482a9 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:28.108357 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 843/1702] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:27 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:28.109273 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f52551-fa51-4e87-af8b-2f407c092600 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:28.139297 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 860/1703] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:13:28 2022] PUT /identity/v3/projects/a31d05c114444ca7948f87e2a5d591a4/users/b16e336cbee349de97bbc0d8c6c73c2e/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:28.140353 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb79d2d-1010-4d8f-8465-ea59401482a9 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:28.144094 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f52551-fa51-4e87-af8b-2f407c092600 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:28.145753 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-0fb79d2d-1010-4d8f-8465-ea59401482a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:28.162011 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:28.178460 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:28.242407 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb79d2d-1010-4d8f-8465-ea59401482a9 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:28.251589 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c713b0-c675-43c2-8ca0-0bf89e2b8456 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:28.253634 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c713b0-c675-43c2-8ca0-0bf89e2b8456 service cinder] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:28.253925 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c713b0-c675-43c2-8ca0-0bf89e2b8456 service cinder] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:28.254197 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c713b0-c675-43c2-8ca0-0bf89e2b8456 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:28.287698 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84137aeb-c1a8-496a-a009-ebf7e3f22052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:28.288596 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84137aeb-c1a8-496a-a009-ebf7e3f22052 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:28.288876 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84137aeb-c1a8-496a-a009-ebf7e3f22052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:28.290474 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84137aeb-c1a8-496a-a009-ebf7e3f22052 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:28.291140 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84137aeb-c1a8-496a-a009-ebf7e3f22052 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:28.292813 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84137aeb-c1a8-496a-a009-ebf7e3f22052 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:28.324678 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 844/1704] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:13:28 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:13:28.331898 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361acf72-4c1c-46e2-a445-e49a3bd73fae None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:28.332506 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361acf72-4c1c-46e2-a445-e49a3bd73fae None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:28.332911 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361acf72-4c1c-46e2-a445-e49a3bd73fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:28.400393 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-361acf72-4c1c-46e2-a445-e49a3bd73fae None None] MFA Rules not processed for user `b16e336cbee349de97bbc0d8c6c73c2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:28.403793 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-361acf72-4c1c-46e2-a445-e49a3bd73fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:28.406839 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c713b0-c675-43c2-8ca0-0bf89e2b8456 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:28.408863 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c713b0-c675-43c2-8ca0-0bf89e2b8456 service cinder] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:28.434928 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 861/1705] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:13:28 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:13:28.436598 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c713b0-c675-43c2-8ca0-0bf89e2b8456 service cinder] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:28.463218 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c713b0-c675-43c2-8ca0-0bf89e2b8456 service cinder] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:28.477098 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950cee89-81ef-471a-8aa7-742ba7880dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:28.477683 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950cee89-81ef-471a-8aa7-742ba7880dc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:28.477965 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950cee89-81ef-471a-8aa7-742ba7880dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:28.478240 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950cee89-81ef-471a-8aa7-742ba7880dc0 admin admin] PATH_INFO: `/v3/projects/d0e45f9536dd42f58b45ec27bf122d94/users/c91e3b58a6464004a9495ff070eeab7a/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:28.520281 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950cee89-81ef-471a-8aa7-742ba7880dc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0e45f9536dd42f58b45ec27bf122d94, user_id=c91e3b58a6464004a9495ff070eeab7a, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:28.525686 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950cee89-81ef-471a-8aa7-742ba7880dc0 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:28.552982 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 862/1706] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:13:28 2022] PUT /identity/v3/projects/d0e45f9536dd42f58b45ec27bf122d94/users/c91e3b58a6464004a9495ff070eeab7a/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:28.564390 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a2de28-fc89-410a-8542-f633f66d49af None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:28.565172 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a2de28-fc89-410a-8542-f633f66d49af None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:28.565697 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a2de28-fc89-410a-8542-f633f66d49af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:28.580843 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 845/1707] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:28 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:28.589191 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361acf72-4c1c-46e2-a445-e49a3bd73fae None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:28.612878 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361acf72-4c1c-46e2-a445-e49a3bd73fae None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:28.648238 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-d0a2de28-fc89-410a-8542-f633f66d49af None None] MFA Rules not processed for user `c91e3b58a6464004a9495ff070eeab7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:28.650755 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-d0a2de28-fc89-410a-8542-f633f66d49af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:28.776787 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb087c24-74c4-416c-96a1-948909b34fad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:28.778398 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb087c24-74c4-416c-96a1-948909b34fad service cinder] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:28.779038 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb087c24-74c4-416c-96a1-948909b34fad service cinder] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:28.779561 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb087c24-74c4-416c-96a1-948909b34fad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:28.869674 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 863/1708] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:28 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:13:28.870340 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a2de28-fc89-410a-8542-f633f66d49af None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:28.871829 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-d0a2de28-fc89-410a-8542-f633f66d49af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:28.906867 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a2de28-fc89-410a-8542-f633f66d49af None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:28.926957 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151051bb-7764-4549-bf69-14a1d0d2df2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:28.927959 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151051bb-7764-4549-bf69-14a1d0d2df2d service cinder] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:28.929667 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151051bb-7764-4549-bf69-14a1d0d2df2d service cinder] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:28.930497 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151051bb-7764-4549-bf69-14a1d0d2df2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:28.935313 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-151051bb-7764-4549-bf69-14a1d0d2df2d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:28.936476 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:28.973846 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb087c24-74c4-416c-96a1-948909b34fad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:28.978242 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb087c24-74c4-416c-96a1-948909b34fad service cinder] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:28.988645 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:28.998897 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 846/1709] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:13:28 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:13:28.999557 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb087c24-74c4-416c-96a1-948909b34fad service cinder] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:29.145698 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151051bb-7764-4549-bf69-14a1d0d2df2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:29.147740 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151051bb-7764-4549-bf69-14a1d0d2df2d service cinder] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:29.172471 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb087c24-74c4-416c-96a1-948909b34fad service cinder] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:29.173113 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 864/1710] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:13:28 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:13:29.174477 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151051bb-7764-4549-bf69-14a1d0d2df2d service cinder] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:29.180673 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201740d7-cabf-47fe-9aa0-ed6cb531b70f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:29.181633 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201740d7-cabf-47fe-9aa0-ed6cb531b70f admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:29.182081 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201740d7-cabf-47fe-9aa0-ed6cb531b70f admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:29.182574 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201740d7-cabf-47fe-9aa0-ed6cb531b70f admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:29.183830 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201740d7-cabf-47fe-9aa0-ed6cb531b70f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:29.187183 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201740d7-cabf-47fe-9aa0-ed6cb531b70f admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:29.220646 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151051bb-7764-4549-bf69-14a1d0d2df2d service cinder] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:29.225180 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189d0fda-a326-4308-8b51-3be526d7e46e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:29.227508 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d0fda-a326-4308-8b51-3be526d7e46e service cinder] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:29.227896 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d0fda-a326-4308-8b51-3be526d7e46e service cinder] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:29.228914 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d0fda-a326-4308-8b51-3be526d7e46e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:29.231148 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-189d0fda-a326-4308-8b51-3be526d7e46e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:29.243666 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 847/1711] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:13:28 2022] POST /identity/v3/projects => generated 399 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:13:29.245589 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e768b6-a87d-4fd6-bf55-13b518400541 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:29.246020 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e768b6-a87d-4fd6-bf55-13b518400541 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:29.246646 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e768b6-a87d-4fd6-bf55-13b518400541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:29.325286 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189d0fda-a326-4308-8b51-3be526d7e46e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:29.326941 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189d0fda-a326-4308-8b51-3be526d7e46e service cinder] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:29.336963 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-d7e768b6-a87d-4fd6-bf55-13b518400541 None None] MFA Rules not processed for user `36fe14a406444624a34a73e2ad60a75c`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:29.341077 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-d7e768b6-a87d-4fd6-bf55-13b518400541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:29.352879 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 865/1712] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:13:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:13:29.356540 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031fba92-0d58-473d-b379-1b080436ecc0 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:29.357474 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031fba92-0d58-473d-b379-1b080436ecc0 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:29.357755 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031fba92-0d58-473d-b379-1b080436ecc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:29.463195 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-031fba92-0d58-473d-b379-1b080436ecc0 None None] MFA Rules not processed for user `9f2e208a67d4443eb00c75c60c54173f`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:29.464458 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-031fba92-0d58-473d-b379-1b080436ecc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:29.478214 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 848/1713] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:29.478214 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e768b6-a87d-4fd6-bf55-13b518400541 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:29.586836 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e768b6-a87d-4fd6-bf55-13b518400541 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:29.601947 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065fa396-7f2a-405c-b8f8-89b600941157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:29.611154 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065fa396-7f2a-405c-b8f8-89b600941157 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:29.611154 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065fa396-7f2a-405c-b8f8-89b600941157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:29.611154 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065fa396-7f2a-405c-b8f8-89b600941157 admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:29.611154 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065fa396-7f2a-405c-b8f8-89b600941157 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:29.613481 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065fa396-7f2a-405c-b8f8-89b600941157 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:29.653493 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:29.690840 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 866/1714] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:29 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:29.692642 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f79ecb3-1aad-41e6-8938-4af3912737d0 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:29.693078 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f79ecb3-1aad-41e6-8938-4af3912737d0 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:29.693483 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f79ecb3-1aad-41e6-8938-4af3912737d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:29.702075 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 849/1715] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:13:29 2022] POST /identity/v3/users => generated 306 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:13:29.703225 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065fa396-7f2a-405c-b8f8-89b600941157 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:29.717481 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065fa396-7f2a-405c-b8f8-89b600941157 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:29.725833 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a900015-14ce-4ea3-8f22-e9f44f94ecab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:29.726735 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a900015-14ce-4ea3-8f22-e9f44f94ecab admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:29.727456 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a900015-14ce-4ea3-8f22-e9f44f94ecab admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:29.727977 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a900015-14ce-4ea3-8f22-e9f44f94ecab admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:29.729252 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a900015-14ce-4ea3-8f22-e9f44f94ecab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:29.731408 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a900015-14ce-4ea3-8f22-e9f44f94ecab admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:29.772062 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 850/1716] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:13:29 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:13:29.772062 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87529127-b1f2-49d4-afcb-da37eef83796 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:29.772062 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87529127-b1f2-49d4-afcb-da37eef83796 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:29.772062 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87529127-b1f2-49d4-afcb-da37eef83796 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:29.774126 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-0f79ecb3-1aad-41e6-8938-4af3912737d0 None None] MFA Rules not processed for user `36fe14a406444624a34a73e2ad60a75c`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:29.777198 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-0f79ecb3-1aad-41e6-8938-4af3912737d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:29.779722 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 851/1717] 10.222.0.42 () {54 vars in 962 bytes} [Thu Sep 8 01:13:29 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:13:29.780935 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87529127-b1f2-49d4-afcb-da37eef83796 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:29.795694 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 867/1718] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:29.803174 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e59de41-2cad-4559-aef3-fff36a3128d4 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:29.804006 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e59de41-2cad-4559-aef3-fff36a3128d4 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:29.805761 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e59de41-2cad-4559-aef3-fff36a3128d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:29.817298 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87529127-b1f2-49d4-afcb-da37eef83796 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:29.834084 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c7b751-c20b-40cd-aab7-fed90a50016d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:29.835034 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7b751-c20b-40cd-aab7-fed90a50016d admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:29.835341 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7b751-c20b-40cd-aab7-fed90a50016d admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:29.835641 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7b751-c20b-40cd-aab7-fed90a50016d admin admin] PATH_INFO: `/v3/projects/9821bb830bde4e83bfe92768f94b7c28/users/481b233c9f4046c89841747625fea9f5/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:29.911035 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c7b751-c20b-40cd-aab7-fed90a50016d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9821bb830bde4e83bfe92768f94b7c28, user_id=481b233c9f4046c89841747625fea9f5, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:29.916209 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-0e59de41-2cad-4559-aef3-fff36a3128d4 None None] MFA Rules not processed for user `ae643cfdcb164aa9b474747cd21aa116`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:29.917499 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-0e59de41-2cad-4559-aef3-fff36a3128d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:29.919935 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c7b751-c20b-40cd-aab7-fed90a50016d admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:29.961357 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 852/1719] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:13:29 2022] PUT /identity/v3/projects/9821bb830bde4e83bfe92768f94b7c28/users/481b233c9f4046c89841747625fea9f5/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:29.964377 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e5d60-67a9-4185-b855-969a02f76c94 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:29.964669 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e5d60-67a9-4185-b855-969a02f76c94 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:29.964973 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e5d60-67a9-4185-b855-969a02f76c94 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:29.968334 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:29.969312 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 853/1720] 10.222.0.42 () {54 vars in 962 bytes} [Thu Sep 8 01:13:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:13:29.971179 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80e5d60-67a9-4185-b855-969a02f76c94 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:30.065466 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 868/1721] 10.222.0.42 () {58 vars in 1059 bytes} [Thu Sep 8 01:13:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:30.068659 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b66fe8-2c6f-4d71-b877-5c0d663f38e2 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:30.069525 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b66fe8-2c6f-4d71-b877-5c0d663f38e2 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:30.070589 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b66fe8-2c6f-4d71-b877-5c0d663f38e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:30.121374 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-61b66fe8-2c6f-4d71-b877-5c0d663f38e2 None None] MFA Rules not processed for user `ae643cfdcb164aa9b474747cd21aa116`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:30.125012 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-61b66fe8-2c6f-4d71-b877-5c0d663f38e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:30.220011 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80e5d60-67a9-4185-b855-969a02f76c94 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:30.233648 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc4b2a-397b-4e0c-96d9-9053fb90d2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:30.238449 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc4b2a-397b-4e0c-96d9-9053fb90d2bd admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:30.238789 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc4b2a-397b-4e0c-96d9-9053fb90d2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:30.239076 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc4b2a-397b-4e0c-96d9-9053fb90d2bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:30.239767 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcc4b2a-397b-4e0c-96d9-9053fb90d2bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:30.242980 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcc4b2a-397b-4e0c-96d9-9053fb90d2bd admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:30.278038 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 854/1722] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:13:29 2022] GET /identity/v3/roles => generated 1420 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:13:30.279459 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc4b2a-397b-4e0c-96d9-9053fb90d2bd admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:30.333287 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 869/1723] 10.222.0.42 () {58 vars in 1059 bytes} [Thu Sep 8 01:13:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:30.335613 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b66fe8-2c6f-4d71-b877-5c0d663f38e2 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:30.372416 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b66fe8-2c6f-4d71-b877-5c0d663f38e2 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:30.385408 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59387485-7964-4a28-a55e-3e4292f7b884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:30.386110 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59387485-7964-4a28-a55e-3e4292f7b884 admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:30.389119 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59387485-7964-4a28-a55e-3e4292f7b884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:30.389461 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59387485-7964-4a28-a55e-3e4292f7b884 admin admin] PATH_INFO: `/v3/projects/9821bb830bde4e83bfe92768f94b7c28/users/481b233c9f4046c89841747625fea9f5/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:30.393699 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59387485-7964-4a28-a55e-3e4292f7b884 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9821bb830bde4e83bfe92768f94b7c28, user_id=481b233c9f4046c89841747625fea9f5, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:30.401598 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59387485-7964-4a28-a55e-3e4292f7b884 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:30.441794 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 870/1724] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:13:30 2022] PUT /identity/v3/projects/9821bb830bde4e83bfe92768f94b7c28/users/481b233c9f4046c89841747625fea9f5/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:30.452876 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59387485-7964-4a28-a55e-3e4292f7b884 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:30.489736 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc4b2a-397b-4e0c-96d9-9053fb90d2bd admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:30.650594 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:30.727127 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98bd7ad-284a-4658-8f9f-340701c92391 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae643cfdcb164aa9b474747cd21aa116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:30.728062 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98bd7ad-284a-4658-8f9f-340701c92391 service nova] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:30.729841 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98bd7ad-284a-4658-8f9f-340701c92391 service nova] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:30.730218 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98bd7ad-284a-4658-8f9f-340701c92391 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:30.765294 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59387485-7964-4a28-a55e-3e4292f7b884 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:30.785453 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a624aaa2-97b0-4d45-bc58-02e53acbefe7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae643cfdcb164aa9b474747cd21aa116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:30.786742 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a624aaa2-97b0-4d45-bc58-02e53acbefe7 service nova] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:30.788227 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a624aaa2-97b0-4d45-bc58-02e53acbefe7 service nova] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:30.789191 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a624aaa2-97b0-4d45-bc58-02e53acbefe7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:30.968624 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:30.974773 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98bd7ad-284a-4658-8f9f-340701c92391 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:30.979349 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98bd7ad-284a-4658-8f9f-340701c92391 service nova] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:31.011216 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 855/1725] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:13:30 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 733 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:13:31.015249 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f10c0-04ec-4a32-b26f-f6498e1de93e None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:31.016308 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f10c0-04ec-4a32-b26f-f6498e1de93e None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:31.018314 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f10c0-04ec-4a32-b26f-f6498e1de93e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:31.036697 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a624aaa2-97b0-4d45-bc58-02e53acbefe7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:31.040915 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a624aaa2-97b0-4d45-bc58-02e53acbefe7 service nova] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:31.072166 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 871/1726] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:13:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 629 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:13:31.094320 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-f22f10c0-04ec-4a32-b26f-f6498e1de93e None None] MFA Rules not processed for user `481b233c9f4046c89841747625fea9f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:31.096796 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-f22f10c0-04ec-4a32-b26f-f6498e1de93e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:31.255691 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 856/1727] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:31 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:31.267788 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a624aaa2-97b0-4d45-bc58-02e53acbefe7 service nova] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:31.500587 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a624aaa2-97b0-4d45-bc58-02e53acbefe7 service nova] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:31.514085 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b3d12b-dcfc-4817-9c50-92b6be54575f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:31.517918 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b3d12b-dcfc-4817-9c50-92b6be54575f service cinder] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:31.520229 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b3d12b-dcfc-4817-9c50-92b6be54575f service cinder] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:31.521366 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b3d12b-dcfc-4817-9c50-92b6be54575f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:31.554788 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b3d12b-dcfc-4817-9c50-92b6be54575f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:31.556478 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b3d12b-dcfc-4817-9c50-92b6be54575f service cinder] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:31.598591 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 872/1728] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:13:31 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:13:31.784874 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22f10c0-04ec-4a32-b26f-f6498e1de93e None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:31.799330 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22f10c0-04ec-4a32-b26f-f6498e1de93e None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:31.825444 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739bfaae-3dfa-4b48-83fa-4e40d85a10c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:31.825444 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739bfaae-3dfa-4b48-83fa-4e40d85a10c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:31.825444 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739bfaae-3dfa-4b48-83fa-4e40d85a10c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:31.825800 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739bfaae-3dfa-4b48-83fa-4e40d85a10c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:32.097140 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739bfaae-3dfa-4b48-83fa-4e40d85a10c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:32.102136 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739bfaae-3dfa-4b48-83fa-4e40d85a10c8 service cinder] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:32.127440 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 857/1729] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:13:31 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:13:32.129548 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67210e27-3203-4c7b-b7cb-8ba6d1275f28 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:32.130020 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67210e27-3203-4c7b-b7cb-8ba6d1275f28 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:32.130100 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67210e27-3203-4c7b-b7cb-8ba6d1275f28 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:32.132431 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 873/1730] 10.222.0.42 () {54 vars in 912 bytes} [Thu Sep 8 01:13:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:13:32.142063 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af80c4-18ac-40d6-bae7-4e087c16f1ec None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:32.142352 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af80c4-18ac-40d6-bae7-4e087c16f1ec None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:32.142626 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af80c4-18ac-40d6-bae7-4e087c16f1ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:32.201788 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-80af80c4-18ac-40d6-bae7-4e087c16f1ec None None] MFA Rules not processed for user `ae643cfdcb164aa9b474747cd21aa116`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:32.206420 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-80af80c4-18ac-40d6-bae7-4e087c16f1ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:32.275124 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 858/1731] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Sep 8 01:13:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:32.291630 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67210e27-3203-4c7b-b7cb-8ba6d1275f28 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:32.293862 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-67210e27-3203-4c7b-b7cb-8ba6d1275f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:32.546153 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67210e27-3203-4c7b-b7cb-8ba6d1275f28 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:32.556596 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0ce245-b0ca-4e33-8f66-ef07448e4a8a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1acfa7a09d34e2a9045c3f3fbea0f52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:32.557232 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ce245-b0ca-4e33-8f66-ef07448e4a8a service glance] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:32.557565 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ce245-b0ca-4e33-8f66-ef07448e4a8a service glance] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:32.557848 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ce245-b0ca-4e33-8f66-ef07448e4a8a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:32.593015 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0ce245-b0ca-4e33-8f66-ef07448e4a8a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:32.601184 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0ce245-b0ca-4e33-8f66-ef07448e4a8a service glance] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:32.654450 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 874/1732] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:13:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:13:33.855899 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e65eeb3-4fc1-4f0a-8c9a-dab4dcce853c None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:33.856542 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e65eeb3-4fc1-4f0a-8c9a-dab4dcce853c None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:33.856716 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e65eeb3-4fc1-4f0a-8c9a-dab4dcce853c None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:33.858497 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 859/1733] 10.222.0.42 () {54 vars in 912 bytes} [Thu Sep 8 01:13:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:13:33.866552 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af88a932-0bb6-4d4e-89e7-1d51e6868fde None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:33.867044 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af88a932-0bb6-4d4e-89e7-1d51e6868fde None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:33.867270 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af88a932-0bb6-4d4e-89e7-1d51e6868fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:33.961198 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-af88a932-0bb6-4d4e-89e7-1d51e6868fde None None] MFA Rules not processed for user `5dbbcc3dbcc745458e97b31c6781ce70`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:33.962940 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-af88a932-0bb6-4d4e-89e7-1d51e6868fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:34.032936 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:34.041484 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 875/1734] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Sep 8 01:13:33 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:13:34.070413 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e65eeb3-4fc1-4f0a-8c9a-dab4dcce853c None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:34.070413 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-1e65eeb3-4fc1-4f0a-8c9a-dab4dcce853c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:34.088386 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:34.097149 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e65eeb3-4fc1-4f0a-8c9a-dab4dcce853c None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:34.106003 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4875f9a-47b2-44dd-9f98-ced0aa8d9584 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:34.106612 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4875f9a-47b2-44dd-9f98-ced0aa8d9584 service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:34.106896 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4875f9a-47b2-44dd-9f98-ced0aa8d9584 service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:34.107163 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4875f9a-47b2-44dd-9f98-ced0aa8d9584 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:34.128646 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4875f9a-47b2-44dd-9f98-ced0aa8d9584 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:34.131015 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4875f9a-47b2-44dd-9f98-ced0aa8d9584 service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:34.159446 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 860/1735] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:13:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:13:35.058861 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c522b3-f0bc-4932-b820-9e4dc134c3a7 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:35.059503 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c522b3-f0bc-4932-b820-9e4dc134c3a7 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:35.059503 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c522b3-f0bc-4932-b820-9e4dc134c3a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:35.174766 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-e0c522b3-f0bc-4932-b820-9e4dc134c3a7 None None] MFA Rules not processed for user `20af4425b7434d2dab7baeacbb845387`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:35.176499 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-e0c522b3-f0bc-4932-b820-9e4dc134c3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:35.441486 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919aacc1-1cb5-4ca3-bca8-f1350599a61f None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:35.442102 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 876/1736] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:35 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:35.442834 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919aacc1-1cb5-4ca3-bca8-f1350599a61f None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:35.443319 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919aacc1-1cb5-4ca3-bca8-f1350599a61f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:35.509751 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-919aacc1-1cb5-4ca3-bca8-f1350599a61f None None] MFA Rules not processed for user `20af4425b7434d2dab7baeacbb845387`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:35.515117 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-919aacc1-1cb5-4ca3-bca8-f1350599a61f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:35.540866 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 861/1737] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:35 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:13:35.553849 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c522b3-f0bc-4932-b820-9e4dc134c3a7 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:35.572055 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c522b3-f0bc-4932-b820-9e4dc134c3a7 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:35.583828 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb694bf-65b1-4287-9b0c-c1ccea6823e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae643cfdcb164aa9b474747cd21aa116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:35.586373 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb694bf-65b1-4287-9b0c-c1ccea6823e5 service nova] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:35.587875 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb694bf-65b1-4287-9b0c-c1ccea6823e5 service nova] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:35.589863 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb694bf-65b1-4287-9b0c-c1ccea6823e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:35.632576 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb694bf-65b1-4287-9b0c-c1ccea6823e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:35.640733 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb694bf-65b1-4287-9b0c-c1ccea6823e5 service nova] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:35.669002 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df193f3e-5a62-4967-b1d0-6b92f6fc25e7 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:35.669427 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df193f3e-5a62-4967-b1d0-6b92f6fc25e7 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:35.670116 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df193f3e-5a62-4967-b1d0-6b92f6fc25e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:35.734554 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 877/1738] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:13:35 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:13:35.743540 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb694bf-65b1-4287-9b0c-c1ccea6823e5 service nova] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:35.745650 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-4fb694bf-65b1-4287-9b0c-c1ccea6823e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:35.828608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-df193f3e-5a62-4967-b1d0-6b92f6fc25e7 None None] MFA Rules not processed for user `3246a8dea80d4d5b8c3fb50d5f1f31c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:35.829987 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-df193f3e-5a62-4967-b1d0-6b92f6fc25e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:35.953405 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 862/1739] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:35.963354 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c936fb0-4c5e-4175-8df5-60d3f28eb814 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:35.963722 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c936fb0-4c5e-4175-8df5-60d3f28eb814 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:35.964072 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c936fb0-4c5e-4175-8df5-60d3f28eb814 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:35.965841 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 863/1740] 10.222.0.42 () {54 vars in 912 bytes} [Thu Sep 8 01:13:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:13:35.966672 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c936fb0-4c5e-4175-8df5-60d3f28eb814 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:35.980815 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-4c936fb0-4c5e-4175-8df5-60d3f28eb814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:35.988588 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb694bf-65b1-4287-9b0c-c1ccea6823e5 service nova] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:35.998649 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3942b653-f0a5-4735-b95a-2deb34ae8845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:36.003115 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3942b653-f0a5-4735-b95a-2deb34ae8845 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:36.007037 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3942b653-f0a5-4735-b95a-2deb34ae8845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:36.007331 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3942b653-f0a5-4735-b95a-2deb34ae8845 admin admin] PATH_INFO: `/v3/users/cbeec441c6154c81a9465a2955f7c8aa` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:36.016989 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3942b653-f0a5-4735-b95a-2deb34ae8845 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbeec441c6154c81a9465a2955f7c8aa)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:36.019107 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3942b653-f0a5-4735-b95a-2deb34ae8845 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:36.138540 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c936fb0-4c5e-4175-8df5-60d3f28eb814 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:36.152283 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cfd19c-56ee-4efc-89f4-cf8c94e04f96 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '301d30bbd3de4cc69e96bb12729442c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:36.152919 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cfd19c-56ee-4efc-89f4-cf8c94e04f96 service placement] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:36.153203 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cfd19c-56ee-4efc-89f4-cf8c94e04f96 service placement] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:36.153476 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cfd19c-56ee-4efc-89f4-cf8c94e04f96 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:36.159251 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-b7cfd19c-56ee-4efc-89f4-cf8c94e04f96 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:36.161139 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-3942b653-f0a5-4735-b95a-2deb34ae8845 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbeec441c6154c81a9465a2955f7c8aa'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:36.187627 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-3942b653-f0a5-4735-b95a-2deb34ae8845 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbeec441c6154c81a9465a2955f7c8aa'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:36.189095 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cfd19c-56ee-4efc-89f4-cf8c94e04f96 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:36.191517 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cfd19c-56ee-4efc-89f4-cf8c94e04f96 service placement] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:36.203455 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-3942b653-f0a5-4735-b95a-2deb34ae8845 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbeec441c6154c81a9465a2955f7c8aa'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:36.211002 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:36.246395 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 864/1741] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:13:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:13:36.247397 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-3942b653-f0a5-4735-b95a-2deb34ae8845 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbeec441c6154c81a9465a2955f7c8aa'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:36.249011 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-3942b653-f0a5-4735-b95a-2deb34ae8845 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbeec441c6154c81a9465a2955f7c8aa'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:36.251475 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33281646-b803-45a7-8dbe-7cd577975e10 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:36.251775 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33281646-b803-45a7-8dbe-7cd577975e10 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:36.252056 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33281646-b803-45a7-8dbe-7cd577975e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:36.275837 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 878/1742] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:13:35 2022] DELETE /identity/v3/users/cbeec441c6154c81a9465a2955f7c8aa => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:36.276878 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3942b653-f0a5-4735-b95a-2deb34ae8845 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:36.278328 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-3942b653-f0a5-4735-b95a-2deb34ae8845 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:36.323582 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-33281646-b803-45a7-8dbe-7cd577975e10 None None] MFA Rules not processed for user `ae643cfdcb164aa9b474747cd21aa116`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:36.324944 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-33281646-b803-45a7-8dbe-7cd577975e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:36.484259 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:36.516786 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 865/1743] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Sep 8 01:13:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:36.520583 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bfc1d7-4bca-4a50-a383-605ca49c37ba None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:36.521118 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bfc1d7-4bca-4a50-a383-605ca49c37ba None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:36.521546 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bfc1d7-4bca-4a50-a383-605ca49c37ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:36.541828 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3942b653-f0a5-4735-b95a-2deb34ae8845 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:36.565478 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0a4d13-29f7-42d6-8cf8-8119e1633d4f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae643cfdcb164aa9b474747cd21aa116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:36.566137 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0a4d13-29f7-42d6-8cf8-8119e1633d4f service nova] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:36.566418 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0a4d13-29f7-42d6-8cf8-8119e1633d4f service nova] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:36.568807 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-19bfc1d7-4bca-4a50-a383-605ca49c37ba None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:36.571173 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-19bfc1d7-4bca-4a50-a383-605ca49c37ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:36.571354 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0a4d13-29f7-42d6-8cf8-8119e1633d4f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:36.583855 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-3c0a4d13-29f7-42d6-8cf8-8119e1633d4f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:36.701369 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0a4d13-29f7-42d6-8cf8-8119e1633d4f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:36.703467 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0a4d13-29f7-42d6-8cf8-8119e1633d4f service nova] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:36.716939 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 879/1744] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:13:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:13:36.718126 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0a4d13-29f7-42d6-8cf8-8119e1633d4f service nova] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:36.720297 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-3c0a4d13-29f7-42d6-8cf8-8119e1633d4f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:36.731438 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 866/1745] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:36.742032 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bfc1d7-4bca-4a50-a383-605ca49c37ba None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:36.744661 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-19bfc1d7-4bca-4a50-a383-605ca49c37ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:36.892547 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0a4d13-29f7-42d6-8cf8-8119e1633d4f service nova] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:36.897393 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bfc1d7-4bca-4a50-a383-605ca49c37ba None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:36.901666 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3095b5-36bb-4dea-9a5a-7b7f089b3611 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:36.902517 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3095b5-36bb-4dea-9a5a-7b7f089b3611 service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:36.902988 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3095b5-36bb-4dea-9a5a-7b7f089b3611 service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:36.903452 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3095b5-36bb-4dea-9a5a-7b7f089b3611 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:36.904363 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a33e12c-03f0-48a9-8152-61378c39fbde service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1acfa7a09d34e2a9045c3f3fbea0f52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:36.905123 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a33e12c-03f0-48a9-8152-61378c39fbde service glance] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:36.905408 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a33e12c-03f0-48a9-8152-61378c39fbde service glance] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:36.905685 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a33e12c-03f0-48a9-8152-61378c39fbde service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:36.918949 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3095b5-36bb-4dea-9a5a-7b7f089b3611 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:36.924350 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3095b5-36bb-4dea-9a5a-7b7f089b3611 service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:36.930401 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a33e12c-03f0-48a9-8152-61378c39fbde service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:36.931894 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a33e12c-03f0-48a9-8152-61378c39fbde service glance] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:36.962206 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 867/1746] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:13:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:13:36.967824 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 880/1747] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:13:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:13:37.442046 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c78ba4-1e77-465f-9bb3-1f332ceeffdb None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:37.449441 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c78ba4-1e77-465f-9bb3-1f332ceeffdb None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:37.449797 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c78ba4-1e77-465f-9bb3-1f332ceeffdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:37.513488 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-04c78ba4-1e77-465f-9bb3-1f332ceeffdb None None] MFA Rules not processed for user `9f2e208a67d4443eb00c75c60c54173f`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:37.519624 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-04c78ba4-1e77-465f-9bb3-1f332ceeffdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:37.642714 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 868/1748] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:37 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:38.113855 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921cae2f-e2af-429c-b70b-ad35d0d652be None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:38.114440 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921cae2f-e2af-429c-b70b-ad35d0d652be None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:38.114440 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921cae2f-e2af-429c-b70b-ad35d0d652be None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:38.117109 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 881/1749] 10.222.0.42 () {54 vars in 912 bytes} [Thu Sep 8 01:13:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:13:38.126931 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23371a0-4836-49cf-b606-bb5100bbd7c0 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:38.127148 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23371a0-4836-49cf-b606-bb5100bbd7c0 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:38.127422 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23371a0-4836-49cf-b606-bb5100bbd7c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:38.176048 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-c23371a0-4836-49cf-b606-bb5100bbd7c0 None None] MFA Rules not processed for user `5dbbcc3dbcc745458e97b31c6781ce70`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:38.177981 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-c23371a0-4836-49cf-b606-bb5100bbd7c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:38.197379 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 869/1750] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Sep 8 01:13:38 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:38.250404 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921cae2f-e2af-429c-b70b-ad35d0d652be None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:38.285924 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921cae2f-e2af-429c-b70b-ad35d0d652be None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:38.301270 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1edc25-318d-4d6e-af3a-72790d68f99a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:38.304778 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1edc25-318d-4d6e-af3a-72790d68f99a service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:38.310602 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1edc25-318d-4d6e-af3a-72790d68f99a service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:38.312443 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1edc25-318d-4d6e-af3a-72790d68f99a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:38.342490 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1edc25-318d-4d6e-af3a-72790d68f99a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:38.347422 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1edc25-318d-4d6e-af3a-72790d68f99a service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:38.382345 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 882/1751] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:13:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:13:38.733289 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23371a0-4836-49cf-b606-bb5100bbd7c0 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:38.739601 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-c23371a0-4836-49cf-b606-bb5100bbd7c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:38.817863 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23371a0-4836-49cf-b606-bb5100bbd7c0 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:38.836107 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71729fc-b6b3-4c63-ad76-e3387aa27065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:38.836506 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71729fc-b6b3-4c63-ad76-e3387aa27065 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:38.836788 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71729fc-b6b3-4c63-ad76-e3387aa27065 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:38.837208 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71729fc-b6b3-4c63-ad76-e3387aa27065 admin admin] PATH_INFO: `/v3/users/c91e3b58a6464004a9495ff070eeab7a` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:38.846442 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71729fc-b6b3-4c63-ad76-e3387aa27065 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c91e3b58a6464004a9495ff070eeab7a)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:38.850249 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71729fc-b6b3-4c63-ad76-e3387aa27065 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:38.994766 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:39.053803 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-c71729fc-b6b3-4c63-ad76-e3387aa27065 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c91e3b58a6464004a9495ff070eeab7a'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:39.166430 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-c71729fc-b6b3-4c63-ad76-e3387aa27065 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c91e3b58a6464004a9495ff070eeab7a'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:39.212595 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-c71729fc-b6b3-4c63-ad76-e3387aa27065 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c91e3b58a6464004a9495ff070eeab7a'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:39.215059 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-c71729fc-b6b3-4c63-ad76-e3387aa27065 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c91e3b58a6464004a9495ff070eeab7a'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:39.232695 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-c71729fc-b6b3-4c63-ad76-e3387aa27065 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c91e3b58a6464004a9495ff070eeab7a'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:39.248181 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 870/1752] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:13:38 2022] DELETE /identity/v3/users/c91e3b58a6464004a9495ff070eeab7a => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:39.253371 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d77dd-eb4e-4153-a18d-f0fd3b621bc3 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:39.254355 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d77dd-eb4e-4153-a18d-f0fd3b621bc3 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:39.254894 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d77dd-eb4e-4153-a18d-f0fd3b621bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:39.294073 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-c47d77dd-eb4e-4153-a18d-f0fd3b621bc3 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:39.297023 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-c47d77dd-eb4e-4153-a18d-f0fd3b621bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:39.339539 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:39.341230 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71729fc-b6b3-4c63-ad76-e3387aa27065 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:39.342963 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-c71729fc-b6b3-4c63-ad76-e3387aa27065 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:39.486752 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 883/1753] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:13:39.488016 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47d77dd-eb4e-4153-a18d-f0fd3b621bc3 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:39.490064 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-c47d77dd-eb4e-4153-a18d-f0fd3b621bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:39.584866 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71729fc-b6b3-4c63-ad76-e3387aa27065 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:39.602173 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa8953-f004-4dac-9f7c-460c67268b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:39.605303 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa8953-f004-4dac-9f7c-460c67268b1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:39.605815 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa8953-f004-4dac-9f7c-460c67268b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:39.607612 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa8953-f004-4dac-9f7c-460c67268b1f admin admin] PATH_INFO: `/v3/projects/c8afdac5e54e44eaadb045a4ca92bf21` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:39.611258 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aa8953-f004-4dac-9f7c-460c67268b1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8afdac5e54e44eaadb045a4ca92bf21)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:39.614822 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aa8953-f004-4dac-9f7c-460c67268b1f admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:39.643191 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47d77dd-eb4e-4153-a18d-f0fd3b621bc3 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:39.674427 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2624235-4ee2-4d44-9414-5219933a39c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:39.675055 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2624235-4ee2-4d44-9414-5219933a39c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:39.675463 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2624235-4ee2-4d44-9414-5219933a39c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:39.675913 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2624235-4ee2-4d44-9414-5219933a39c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:39.711655 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2624235-4ee2-4d44-9414-5219933a39c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:39.714819 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2624235-4ee2-4d44-9414-5219933a39c7 service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:39.745752 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 884/1754] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:13:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:13:39.873049 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-b0aa8953-f004-4dac-9f7c-460c67268b1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8afdac5e54e44eaadb045a4ca92bf21', 'request_id': 'req-b0aa8953-f004-4dac-9f7c-460c67268b1f'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:39.896683 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-b0aa8953-f004-4dac-9f7c-460c67268b1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8afdac5e54e44eaadb045a4ca92bf21', 'request_id': 'req-b0aa8953-f004-4dac-9f7c-460c67268b1f'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:39.914775 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-b0aa8953-f004-4dac-9f7c-460c67268b1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8afdac5e54e44eaadb045a4ca92bf21', 'request_id': 'req-b0aa8953-f004-4dac-9f7c-460c67268b1f'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:39.947614 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-b0aa8953-f004-4dac-9f7c-460c67268b1f admin admin] The token cache is being invalidate because project c8afdac5e54e44eaadb045a4ca92bf21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81661) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:13:39.948436 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-b0aa8953-f004-4dac-9f7c-460c67268b1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:39.955187 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-b0aa8953-f004-4dac-9f7c-460c67268b1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:39.957499 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 871/1755] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:13:39 2022] DELETE /identity/v3/projects/c8afdac5e54e44eaadb045a4ca92bf21 => generated 0 bytes in 616 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:39.961287 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2624235-4ee2-4d44-9414-5219933a39c7 service neutron] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:39.980884 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2624235-4ee2-4d44-9414-5219933a39c7 service neutron] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:40.233182 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c62fa-039a-4eb9-8ebe-96f482872c00 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:40.233182 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c62fa-039a-4eb9-8ebe-96f482872c00 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:40.233182 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c62fa-039a-4eb9-8ebe-96f482872c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:40.239529 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29952023-a22a-4966-8593-1a5c40517d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:40.240310 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29952023-a22a-4966-8593-1a5c40517d73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:40.240659 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29952023-a22a-4966-8593-1a5c40517d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:40.241005 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29952023-a22a-4966-8593-1a5c40517d73 admin admin] PATH_INFO: `/v3/users/66492b2909294b0cafc1261b185d35c6` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:40.242843 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29952023-a22a-4966-8593-1a5c40517d73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66492b2909294b0cafc1261b185d35c6)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:40.246776 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29952023-a22a-4966-8593-1a5c40517d73 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:40.273737 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-b51c62fa-039a-4eb9-8ebe-96f482872c00 None None] MFA Rules not processed for user `20af4425b7434d2dab7baeacbb845387`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:40.275837 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-b51c62fa-039a-4eb9-8ebe-96f482872c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:40.351272 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:40.419281 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-29952023-a22a-4966-8593-1a5c40517d73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66492b2909294b0cafc1261b185d35c6'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:40.449138 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-29952023-a22a-4966-8593-1a5c40517d73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66492b2909294b0cafc1261b185d35c6'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:40.450533 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:40.462919 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-29952023-a22a-4966-8593-1a5c40517d73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66492b2909294b0cafc1261b185d35c6'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:40.480517 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-29952023-a22a-4966-8593-1a5c40517d73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66492b2909294b0cafc1261b185d35c6'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:40.483246 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-29952023-a22a-4966-8593-1a5c40517d73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66492b2909294b0cafc1261b185d35c6'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:40.483959 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 872/1756] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:40 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:40.533427 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 885/1757] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:13:39 2022] DELETE /identity/v3/users/66492b2909294b0cafc1261b185d35c6 => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:40.892966 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53937c34-bc65-4a86-bf0c-3791d743cc9b None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:40.893517 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53937c34-bc65-4a86-bf0c-3791d743cc9b None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:40.893517 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53937c34-bc65-4a86-bf0c-3791d743cc9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:41.019200 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-53937c34-bc65-4a86-bf0c-3791d743cc9b None None] MFA Rules not processed for user `de75fd7d7721433684cafa2e6d3faaae`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:41.025959 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-53937c34-bc65-4a86-bf0c-3791d743cc9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:41.199558 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 873/1758] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Sep 8 01:13:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:41.408430 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29952023-a22a-4966-8593-1a5c40517d73 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:41.410056 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-29952023-a22a-4966-8593-1a5c40517d73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:41.682233 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29952023-a22a-4966-8593-1a5c40517d73 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:41.690663 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c897428-229b-4a60-be24-27d6c92822f2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8add26872fe6479aa9d7de155a17195a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:41.691555 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c897428-229b-4a60-be24-27d6c92822f2 service swift] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:41.691990 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c897428-229b-4a60-be24-27d6c92822f2 service swift] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:41.692437 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c897428-229b-4a60-be24-27d6c92822f2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:41.703105 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c897428-229b-4a60-be24-27d6c92822f2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:41.706488 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c897428-229b-4a60-be24-27d6c92822f2 service swift] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:41.716648 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 886/1759] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Sep 8 01:13:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 01:13:42.245357 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53937c34-bc65-4a86-bf0c-3791d743cc9b None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:42.248039 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-53937c34-bc65-4a86-bf0c-3791d743cc9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:42.496831 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53937c34-bc65-4a86-bf0c-3791d743cc9b None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:42.503985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700386f4-b7c4-4a8c-9f52-64c6f5c92869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:42.505827 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700386f4-b7c4-4a8c-9f52-64c6f5c92869 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:42.506265 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700386f4-b7c4-4a8c-9f52-64c6f5c92869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:42.506693 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700386f4-b7c4-4a8c-9f52-64c6f5c92869 admin admin] PATH_INFO: `/v3/projects/d0e45f9536dd42f58b45ec27bf122d94` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:42.508788 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700386f4-b7c4-4a8c-9f52-64c6f5c92869 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0e45f9536dd42f58b45ec27bf122d94)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:42.511701 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700386f4-b7c4-4a8c-9f52-64c6f5c92869 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:42.576927 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c897428-229b-4a60-be24-27d6c92822f2 service swift] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:42.579947 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-3c897428-229b-4a60-be24-27d6c92822f2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:42.636435 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c897428-229b-4a60-be24-27d6c92822f2 service swift] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:42.670354 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7ec12d-9311-48c1-8f12-37f9953446ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:42.670894 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ec12d-9311-48c1-8f12-37f9953446ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:42.679140 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ec12d-9311-48c1-8f12-37f9953446ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:42.680680 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ec12d-9311-48c1-8f12-37f9953446ac admin admin] PATH_INFO: `/v3/projects/6cba181189ba42f097c59ec57be303fa` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:42.685408 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7ec12d-9311-48c1-8f12-37f9953446ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cba181189ba42f097c59ec57be303fa)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:42.699593 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7ec12d-9311-48c1-8f12-37f9953446ac admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:42.709898 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-700386f4-b7c4-4a8c-9f52-64c6f5c92869 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0e45f9536dd42f58b45ec27bf122d94', 'request_id': 'req-700386f4-b7c4-4a8c-9f52-64c6f5c92869'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:42.725875 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-700386f4-b7c4-4a8c-9f52-64c6f5c92869 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0e45f9536dd42f58b45ec27bf122d94', 'request_id': 'req-700386f4-b7c4-4a8c-9f52-64c6f5c92869'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:42.734166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:42.763396 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-700386f4-b7c4-4a8c-9f52-64c6f5c92869 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0e45f9536dd42f58b45ec27bf122d94', 'request_id': 'req-700386f4-b7c4-4a8c-9f52-64c6f5c92869'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:42.797494 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-700386f4-b7c4-4a8c-9f52-64c6f5c92869 admin admin] The token cache is being invalidate because project d0e45f9536dd42f58b45ec27bf122d94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81661) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:13:42.797834 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-700386f4-b7c4-4a8c-9f52-64c6f5c92869 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:42.798847 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-700386f4-b7c4-4a8c-9f52-64c6f5c92869 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:42.805451 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 874/1760] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:13:42 2022] DELETE /identity/v3/projects/d0e45f9536dd42f58b45ec27bf122d94 => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:42.807312 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700386f4-b7c4-4a8c-9f52-64c6f5c92869 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:42.810462 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-700386f4-b7c4-4a8c-9f52-64c6f5c92869 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:42.945386 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700386f4-b7c4-4a8c-9f52-64c6f5c92869 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:42.965627 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b7d1e-2554-4065-b259-b54189d4e2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:42.968606 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b7d1e-2554-4065-b259-b54189d4e2b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:42.968955 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b7d1e-2554-4065-b259-b54189d4e2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:42.972215 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b7d1e-2554-4065-b259-b54189d4e2b6 admin admin] PATH_INFO: `/v3/users/481b233c9f4046c89841747625fea9f5` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:42.975446 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64b7d1e-2554-4065-b259-b54189d4e2b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=481b233c9f4046c89841747625fea9f5)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:42.979450 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64b7d1e-2554-4065-b259-b54189d4e2b6 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:43.131714 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-8e7ec12d-9311-48c1-8f12-37f9953446ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cba181189ba42f097c59ec57be303fa', 'request_id': 'req-8e7ec12d-9311-48c1-8f12-37f9953446ac'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:43.187625 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-b64b7d1e-2554-4065-b259-b54189d4e2b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '481b233c9f4046c89841747625fea9f5'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:43.190409 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-8e7ec12d-9311-48c1-8f12-37f9953446ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cba181189ba42f097c59ec57be303fa', 'request_id': 'req-8e7ec12d-9311-48c1-8f12-37f9953446ac'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:43.230841 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-8e7ec12d-9311-48c1-8f12-37f9953446ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cba181189ba42f097c59ec57be303fa', 'request_id': 'req-8e7ec12d-9311-48c1-8f12-37f9953446ac'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:43.269240 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-b64b7d1e-2554-4065-b259-b54189d4e2b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '481b233c9f4046c89841747625fea9f5'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:43.288514 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-8e7ec12d-9311-48c1-8f12-37f9953446ac admin admin] The token cache is being invalidate because project 6cba181189ba42f097c59ec57be303fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:13:43.294453 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-8e7ec12d-9311-48c1-8f12-37f9953446ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:43.298185 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-8e7ec12d-9311-48c1-8f12-37f9953446ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:43.310293 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 887/1761] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:13:42 2022] DELETE /identity/v3/projects/6cba181189ba42f097c59ec57be303fa => generated 0 bytes in 731 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:13:43.310601 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-b64b7d1e-2554-4065-b259-b54189d4e2b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '481b233c9f4046c89841747625fea9f5'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:43.312636 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-b64b7d1e-2554-4065-b259-b54189d4e2b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '481b233c9f4046c89841747625fea9f5'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:43.315807 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36295aa-838c-488e-8165-3c36519a38e7 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:43.317412 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36295aa-838c-488e-8165-3c36519a38e7 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:43.317694 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36295aa-838c-488e-8165-3c36519a38e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:43.339416 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-b64b7d1e-2554-4065-b259-b54189d4e2b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '481b233c9f4046c89841747625fea9f5'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:43.369619 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 875/1762] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:13:42 2022] DELETE /identity/v3/users/481b233c9f4046c89841747625fea9f5 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:43.399204 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-c36295aa-838c-488e-8165-3c36519a38e7 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:43.403182 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-c36295aa-838c-488e-8165-3c36519a38e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:43.532420 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:43.707959 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 888/1763] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:43.710770 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad277afa-e9e6-4f56-a84a-1eb4bd0635af None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:43.711290 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad277afa-e9e6-4f56-a84a-1eb4bd0635af None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:43.711716 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad277afa-e9e6-4f56-a84a-1eb4bd0635af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:43.757284 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-ad277afa-e9e6-4f56-a84a-1eb4bd0635af None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:43.761539 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-ad277afa-e9e6-4f56-a84a-1eb4bd0635af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:43.804775 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 876/1764] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:43.807088 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36295aa-838c-488e-8165-3c36519a38e7 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:43.821603 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36295aa-838c-488e-8165-3c36519a38e7 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:43.861325 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549364e7-98c9-4ca1-b704-c91dc8f33a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:43.862194 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549364e7-98c9-4ca1-b704-c91dc8f33a43 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:43.862866 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549364e7-98c9-4ca1-b704-c91dc8f33a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:43.863745 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549364e7-98c9-4ca1-b704-c91dc8f33a43 admin admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:43.864648 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549364e7-98c9-4ca1-b704-c91dc8f33a43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:13:43.865159 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549364e7-98c9-4ca1-b704-c91dc8f33a43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:43.867036 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549364e7-98c9-4ca1-b704-c91dc8f33a43 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:43.924105 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad277afa-e9e6-4f56-a84a-1eb4bd0635af None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:43.940391 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad277afa-e9e6-4f56-a84a-1eb4bd0635af None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:43.940674 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 889/1765] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:13:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:13:43.949394 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c621d4f1-db2d-478f-b474-72efd160cfbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:43.952893 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c621d4f1-db2d-478f-b474-72efd160cfbc admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:43.953622 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c621d4f1-db2d-478f-b474-72efd160cfbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:43.954707 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c621d4f1-db2d-478f-b474-72efd160cfbc admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:43.956737 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c621d4f1-db2d-478f-b474-72efd160cfbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:43.960674 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c621d4f1-db2d-478f-b474-72efd160cfbc admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:44.064938 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 877/1766] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:13:43 2022] POST /identity/v3/projects => generated 373 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 01:13:44.067255 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549364e7-98c9-4ca1-b704-c91dc8f33a43 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:44.079837 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549364e7-98c9-4ca1-b704-c91dc8f33a43 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:44.254405 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b3517-5276-4128-970d-6ab178e931a2 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:44.254743 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b3517-5276-4128-970d-6ab178e931a2 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:44.254961 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b3517-5276-4128-970d-6ab178e931a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:44.299524 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b6a61f-f83b-471e-b241-954a5d4ff57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:44.300224 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b6a61f-f83b-471e-b241-954a5d4ff57f admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:44.300511 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b6a61f-f83b-471e-b241-954a5d4ff57f admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:44.302244 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-0e3b3517-5276-4128-970d-6ab178e931a2 None None] MFA Rules not processed for user `5dbbcc3dbcc745458e97b31c6781ce70`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:44.303769 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b6a61f-f83b-471e-b241-954a5d4ff57f admin admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:44.304511 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b6a61f-f83b-471e-b241-954a5d4ff57f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:44.306520 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b6a61f-f83b-471e-b241-954a5d4ff57f admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:44.309242 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-0e3b3517-5276-4128-970d-6ab178e931a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:44.406419 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 890/1767] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:13:44 2022] POST /identity/v3/users => generated 287 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:13:44.415691 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b6a61f-f83b-471e-b241-954a5d4ff57f admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:44.480494 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b6a61f-f83b-471e-b241-954a5d4ff57f admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:44.499069 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:44.517055 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c0ecac-c489-437a-926d-c3e59caa1e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:44.517974 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0ecac-c489-437a-926d-c3e59caa1e29 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:44.520702 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0ecac-c489-437a-926d-c3e59caa1e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:44.521163 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0ecac-c489-437a-926d-c3e59caa1e29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:44.522007 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c0ecac-c489-437a-926d-c3e59caa1e29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:44.527047 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c0ecac-c489-437a-926d-c3e59caa1e29 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:44.567784 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 878/1768] 10.222.0.46 () {58 vars in 1015 bytes} [Thu Sep 8 01:13:44 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:13:44.599935 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 891/1769] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:13:44.601013 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b3517-5276-4128-970d-6ab178e931a2 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:44.631588 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b3517-5276-4128-970d-6ab178e931a2 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:44.652023 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be5cda5-34b4-4b07-8dba-052efbeea86e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:44.656848 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be5cda5-34b4-4b07-8dba-052efbeea86e admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:44.662953 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be5cda5-34b4-4b07-8dba-052efbeea86e admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:44.663811 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be5cda5-34b4-4b07-8dba-052efbeea86e admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:44.666884 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be5cda5-34b4-4b07-8dba-052efbeea86e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:44.673621 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be5cda5-34b4-4b07-8dba-052efbeea86e admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:44.720848 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c0ecac-c489-437a-926d-c3e59caa1e29 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:44.725658 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-e3c0ecac-c489-437a-926d-c3e59caa1e29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:44.762412 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 879/1770] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:13:44.763780 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c0ecac-c489-437a-926d-c3e59caa1e29 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:44.765600 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be5cda5-34b4-4b07-8dba-052efbeea86e admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:44.796703 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c28297-bb94-4b52-8dd7-0bcf9faff16a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:44.797363 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c28297-bb94-4b52-8dd7-0bcf9faff16a service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:44.802271 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c28297-bb94-4b52-8dd7-0bcf9faff16a service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:44.805822 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c28297-bb94-4b52-8dd7-0bcf9faff16a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:44.845430 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c28297-bb94-4b52-8dd7-0bcf9faff16a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:44.847122 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c28297-bb94-4b52-8dd7-0bcf9faff16a service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:44.862368 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be5cda5-34b4-4b07-8dba-052efbeea86e admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:44.882755 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9299e12f-8951-4f5c-a322-d947667bdc52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:44.886178 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299e12f-8951-4f5c-a322-d947667bdc52 admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:44.887401 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299e12f-8951-4f5c-a322-d947667bdc52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:44.888516 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 892/1771] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:13:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:13:44.891136 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299e12f-8951-4f5c-a322-d947667bdc52 admin admin] PATH_INFO: `/v3/projects/ce6f9e2e136c4ea4b76c77acd5a7b928/users/d7f03f8a45164e9eb5f61b2520e35694/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:44.957377 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9299e12f-8951-4f5c-a322-d947667bdc52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce6f9e2e136c4ea4b76c77acd5a7b928, user_id=d7f03f8a45164e9eb5f61b2520e35694, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:44.965342 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9299e12f-8951-4f5c-a322-d947667bdc52 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:45.030841 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 880/1772] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:13:44 2022] PUT /identity/v3/projects/ce6f9e2e136c4ea4b76c77acd5a7b928/users/d7f03f8a45164e9eb5f61b2520e35694/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:45.041641 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d94bc-7e5a-4685-b1fa-98be4fb5cb48 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:45.042094 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d94bc-7e5a-4685-b1fa-98be4fb5cb48 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:45.042531 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d94bc-7e5a-4685-b1fa-98be4fb5cb48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:45.120546 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-b42d94bc-7e5a-4685-b1fa-98be4fb5cb48 None None] MFA Rules not processed for user `d7f03f8a45164e9eb5f61b2520e35694`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:45.121828 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-b42d94bc-7e5a-4685-b1fa-98be4fb5cb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:45.355819 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 893/1773] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:45.356849 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9299e12f-8951-4f5c-a322-d947667bdc52 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:45.648190 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9299e12f-8951-4f5c-a322-d947667bdc52 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:45.666400 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0602261-9a81-474e-8dcb-28e5cfb3343a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:45.668009 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0602261-9a81-474e-8dcb-28e5cfb3343a admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:45.668835 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0602261-9a81-474e-8dcb-28e5cfb3343a admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:45.669369 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0602261-9a81-474e-8dcb-28e5cfb3343a admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:45.670292 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0602261-9a81-474e-8dcb-28e5cfb3343a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:45.672856 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0602261-9a81-474e-8dcb-28e5cfb3343a admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:45.713057 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 881/1774] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:13:45 2022] POST /identity/v3/projects => generated 373 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 01:13:45.717183 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d94bc-7e5a-4685-b1fa-98be4fb5cb48 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:45.732350 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d94bc-7e5a-4685-b1fa-98be4fb5cb48 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:45.758727 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:45.839807 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a149f1f3-3393-4301-a53e-1ce173c24068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:45.841591 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149f1f3-3393-4301-a53e-1ce173c24068 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:45.843509 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149f1f3-3393-4301-a53e-1ce173c24068 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:45.844613 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149f1f3-3393-4301-a53e-1ce173c24068 admin admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:45.846645 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a149f1f3-3393-4301-a53e-1ce173c24068 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:45.853228 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a149f1f3-3393-4301-a53e-1ce173c24068 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:45.897933 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b04b45-9a6e-4005-b3a0-04c2b77a18e1 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:45.898436 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b04b45-9a6e-4005-b3a0-04c2b77a18e1 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:45.903685 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b04b45-9a6e-4005-b3a0-04c2b77a18e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:45.966760 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-a4b04b45-9a6e-4005-b3a0-04c2b77a18e1 None None] MFA Rules not processed for user `3246a8dea80d4d5b8c3fb50d5f1f31c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:45.977684 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-a4b04b45-9a6e-4005-b3a0-04c2b77a18e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:45.993898 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 894/1775] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:13:45 2022] POST /identity/v3/users => generated 293 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:13:45.998339 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a149f1f3-3393-4301-a53e-1ce173c24068 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:46.053745 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a149f1f3-3393-4301-a53e-1ce173c24068 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:46.083007 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6264e-cdca-4e0d-bf69-0bb7aaa1dfa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:46.085669 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6264e-cdca-4e0d-bf69-0bb7aaa1dfa9 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:46.086326 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6264e-cdca-4e0d-bf69-0bb7aaa1dfa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:46.087409 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6264e-cdca-4e0d-bf69-0bb7aaa1dfa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:46.090786 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a6264e-cdca-4e0d-bf69-0bb7aaa1dfa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:46.101314 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a6264e-cdca-4e0d-bf69-0bb7aaa1dfa9 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:46.167128 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 895/1776] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:13:45 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:13:46.172853 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6264e-cdca-4e0d-bf69-0bb7aaa1dfa9 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:46.187062 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 882/1777] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:46.225463 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6264e-cdca-4e0d-bf69-0bb7aaa1dfa9 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:46.241869 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15138db4-92a3-4759-ae46-7c72a0eb6dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:46.242516 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15138db4-92a3-4759-ae46-7c72a0eb6dfe admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:46.242792 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15138db4-92a3-4759-ae46-7c72a0eb6dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:46.243065 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15138db4-92a3-4759-ae46-7c72a0eb6dfe admin admin] PATH_INFO: `/v3/projects/46a33e5bf04848f78af61abe424cf3f1/users/85c96ff95f92435d8df7a9950be3e076/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:46.285021 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15138db4-92a3-4759-ae46-7c72a0eb6dfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=46a33e5bf04848f78af61abe424cf3f1, user_id=85c96ff95f92435d8df7a9950be3e076, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:46.291051 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15138db4-92a3-4759-ae46-7c72a0eb6dfe admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:46.329307 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b04b45-9a6e-4005-b3a0-04c2b77a18e1 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:46.335617 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 896/1778] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:13:46 2022] PUT /identity/v3/projects/46a33e5bf04848f78af61abe424cf3f1/users/85c96ff95f92435d8df7a9950be3e076/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:13:46.356764 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b04b45-9a6e-4005-b3a0-04c2b77a18e1 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:46.498244 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:46.551709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57930464-b02c-431b-b257-02779ebdd078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:46.552522 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57930464-b02c-431b-b257-02779ebdd078 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:46.552979 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57930464-b02c-431b-b257-02779ebdd078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:46.553402 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57930464-b02c-431b-b257-02779ebdd078 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:46.554210 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57930464-b02c-431b-b257-02779ebdd078 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:46.556250 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57930464-b02c-431b-b257-02779ebdd078 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:46.590820 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 883/1779] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:13:46.595845 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15138db4-92a3-4759-ae46-7c72a0eb6dfe admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:46.650859 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15138db4-92a3-4759-ae46-7c72a0eb6dfe admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:46.674485 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0475d2de-008a-4b81-bd04-fb1a80189ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:46.675956 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475d2de-008a-4b81-bd04-fb1a80189ca9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:46.676877 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475d2de-008a-4b81-bd04-fb1a80189ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:46.677387 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475d2de-008a-4b81-bd04-fb1a80189ca9 admin admin] PATH_INFO: `/v3/projects/46a33e5bf04848f78af61abe424cf3f1/users/85c96ff95f92435d8df7a9950be3e076/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:46.680344 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0475d2de-008a-4b81-bd04-fb1a80189ca9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46a33e5bf04848f78af61abe424cf3f1, user_id=85c96ff95f92435d8df7a9950be3e076, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:46.684895 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0475d2de-008a-4b81-bd04-fb1a80189ca9 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:46.708301 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 897/1780] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:13:46 2022] PUT /identity/v3/projects/46a33e5bf04848f78af61abe424cf3f1/users/85c96ff95f92435d8df7a9950be3e076/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:46.719655 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f2b5a-e0bb-4672-b434-2ef658956155 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:46.719901 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f2b5a-e0bb-4672-b434-2ef658956155 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:46.720253 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f2b5a-e0bb-4672-b434-2ef658956155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:46.805126 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-b26f2b5a-e0bb-4672-b434-2ef658956155 None None] MFA Rules not processed for user `85c96ff95f92435d8df7a9950be3e076`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:46.806491 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-b26f2b5a-e0bb-4672-b434-2ef658956155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:46.853331 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0475d2de-008a-4b81-bd04-fb1a80189ca9 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:46.855524 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-0475d2de-008a-4b81-bd04-fb1a80189ca9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:47.198075 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 884/1781] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:46 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 479 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:47.214505 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0475d2de-008a-4b81-bd04-fb1a80189ca9 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:47.216549 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f2b5a-e0bb-4672-b434-2ef658956155 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:47.221677 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-b26f2b5a-e0bb-4672-b434-2ef658956155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:47.253108 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9827d47c-d4a8-46f8-9881-324bcd34db10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:47.253856 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9827d47c-d4a8-46f8-9881-324bcd34db10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:47.254362 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9827d47c-d4a8-46f8-9881-324bcd34db10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:47.254839 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9827d47c-d4a8-46f8-9881-324bcd34db10 admin admin] PATH_INFO: `/v3/projects/9821bb830bde4e83bfe92768f94b7c28` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:47.257358 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9827d47c-d4a8-46f8-9881-324bcd34db10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9821bb830bde4e83bfe92768f94b7c28)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:47.259592 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9827d47c-d4a8-46f8-9881-324bcd34db10 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:47.455887 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f2b5a-e0bb-4672-b434-2ef658956155 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:47.467787 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88edc665-c896-4966-9ba5-453f9d755e59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:47.473192 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edc665-c896-4966-9ba5-453f9d755e59 service cinder] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:47.473734 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edc665-c896-4966-9ba5-453f9d755e59 service cinder] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:47.474617 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edc665-c896-4966-9ba5-453f9d755e59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:47.515279 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-9827d47c-d4a8-46f8-9881-324bcd34db10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9821bb830bde4e83bfe92768f94b7c28', 'request_id': 'req-9827d47c-d4a8-46f8-9881-324bcd34db10'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:47.516312 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88edc665-c896-4966-9ba5-453f9d755e59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:47.518537 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88edc665-c896-4966-9ba5-453f9d755e59 service cinder] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:47.545305 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-9827d47c-d4a8-46f8-9881-324bcd34db10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9821bb830bde4e83bfe92768f94b7c28', 'request_id': 'req-9827d47c-d4a8-46f8-9881-324bcd34db10'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:47.548822 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 885/1782] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:13:47 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:13:47.560844 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:47.574912 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-9827d47c-d4a8-46f8-9881-324bcd34db10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9821bb830bde4e83bfe92768f94b7c28', 'request_id': 'req-9827d47c-d4a8-46f8-9881-324bcd34db10'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:47.620065 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-9827d47c-d4a8-46f8-9881-324bcd34db10 admin admin] The token cache is being invalidate because project 9821bb830bde4e83bfe92768f94b7c28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:13:47.620511 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-9827d47c-d4a8-46f8-9881-324bcd34db10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:47.621341 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-9827d47c-d4a8-46f8-9881-324bcd34db10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:47.625078 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 898/1783] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:13:46 2022] DELETE /identity/v3/projects/9821bb830bde4e83bfe92768f94b7c28 => generated 0 bytes in 773 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:13:47.630490 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b616f2d1-97df-4645-ad42-aca902927b27 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:47.631327 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b616f2d1-97df-4645-ad42-aca902927b27 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:47.631799 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b616f2d1-97df-4645-ad42-aca902927b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:47.711068 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-b616f2d1-97df-4645-ad42-aca902927b27 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:47.714576 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-b616f2d1-97df-4645-ad42-aca902927b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:47.933196 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 886/1784] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:47.945628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80c753-276c-4566-9ee2-b0f31f1391e5 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:47.946141 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80c753-276c-4566-9ee2-b0f31f1391e5 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:47.946538 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80c753-276c-4566-9ee2-b0f31f1391e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:47.994508 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-0d80c753-276c-4566-9ee2-b0f31f1391e5 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:47.998492 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-0d80c753-276c-4566-9ee2-b0f31f1391e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:48.026240 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 899/1785] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:48.031122 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b616f2d1-97df-4645-ad42-aca902927b27 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:48.051689 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b616f2d1-97df-4645-ad42-aca902927b27 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:48.064695 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3994da-8639-48e7-b832-1def7f1ba1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:48.065481 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3994da-8639-48e7-b832-1def7f1ba1db admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:48.065820 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3994da-8639-48e7-b832-1def7f1ba1db admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:48.066098 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3994da-8639-48e7-b832-1def7f1ba1db admin admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:48.066986 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3994da-8639-48e7-b832-1def7f1ba1db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:13:48.067324 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3994da-8639-48e7-b832-1def7f1ba1db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:48.069044 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3994da-8639-48e7-b832-1def7f1ba1db admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:48.070960 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:48.100029 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 887/1786] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:13:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:13:48.105765 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d80c753-276c-4566-9ee2-b0f31f1391e5 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:48.122575 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d80c753-276c-4566-9ee2-b0f31f1391e5 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:48.142091 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba2aa81-e3d0-4b71-9799-89f8f59ab091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:48.143171 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2aa81-e3d0-4b71-9799-89f8f59ab091 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:48.143472 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2aa81-e3d0-4b71-9799-89f8f59ab091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:48.143747 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2aa81-e3d0-4b71-9799-89f8f59ab091 admin admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:48.144460 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba2aa81-e3d0-4b71-9799-89f8f59ab091 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:48.146635 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba2aa81-e3d0-4b71-9799-89f8f59ab091 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:48.238618 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 900/1787] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:13:48 2022] POST /identity/v3/projects => generated 383 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:13:48.240239 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3994da-8639-48e7-b832-1def7f1ba1db admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:48.262826 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3994da-8639-48e7-b832-1def7f1ba1db admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:48.504981 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e8c229-b7ea-4f42-964e-64c2f14cc73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:48.505736 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e8c229-b7ea-4f42-964e-64c2f14cc73a admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:48.508531 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e8c229-b7ea-4f42-964e-64c2f14cc73a admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:48.511751 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e8c229-b7ea-4f42-964e-64c2f14cc73a admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:48.513773 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e8c229-b7ea-4f42-964e-64c2f14cc73a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:48.521271 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e8c229-b7ea-4f42-964e-64c2f14cc73a admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:48.682543 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 888/1788] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:13:48 2022] POST /identity/v3/users => generated 292 bytes in 443 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:13:48.686886 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba2aa81-e3d0-4b71-9799-89f8f59ab091 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:48.708173 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba2aa81-e3d0-4b71-9799-89f8f59ab091 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:48.739518 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef0b053-8df5-4077-9c44-be95454e4302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:48.740207 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef0b053-8df5-4077-9c44-be95454e4302 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:48.740475 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef0b053-8df5-4077-9c44-be95454e4302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:48.740754 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef0b053-8df5-4077-9c44-be95454e4302 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:48.741412 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef0b053-8df5-4077-9c44-be95454e4302 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:48.743022 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef0b053-8df5-4077-9c44-be95454e4302 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:48.811002 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 901/1789] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:13:48 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:13:48.815268 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e8c229-b7ea-4f42-964e-64c2f14cc73a admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:48.830397 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e8c229-b7ea-4f42-964e-64c2f14cc73a admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:48.857721 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf287581-d996-4a73-9f5f-cfe8e8944ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:48.865617 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf287581-d996-4a73-9f5f-cfe8e8944ac4 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:48.865617 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf287581-d996-4a73-9f5f-cfe8e8944ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:48.865617 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf287581-d996-4a73-9f5f-cfe8e8944ac4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:48.865617 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf287581-d996-4a73-9f5f-cfe8e8944ac4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:48.869517 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf287581-d996-4a73-9f5f-cfe8e8944ac4 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:48.933774 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 889/1790] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:13:48 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:13:48.937161 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef0b053-8df5-4077-9c44-be95454e4302 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:48.954399 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef0b053-8df5-4077-9c44-be95454e4302 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:48.987465 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78c8d04-70cc-4210-825e-b02d963eb792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:48.991989 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c8d04-70cc-4210-825e-b02d963eb792 admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:48.994119 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c8d04-70cc-4210-825e-b02d963eb792 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:48.994691 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c8d04-70cc-4210-825e-b02d963eb792 admin admin] PATH_INFO: `/v3/projects/39c5a342a8084982a835660cd17ae080/users/64591b8931e84f85b12d587c2ff19ce8/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:49.062825 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78c8d04-70cc-4210-825e-b02d963eb792 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39c5a342a8084982a835660cd17ae080, user_id=64591b8931e84f85b12d587c2ff19ce8, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:49.070636 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78c8d04-70cc-4210-825e-b02d963eb792 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:49.116357 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 902/1791] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:13:48 2022] PUT /identity/v3/projects/39c5a342a8084982a835660cd17ae080/users/64591b8931e84f85b12d587c2ff19ce8/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:49.126547 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeee02b-a8c7-49f1-8322-af17231be6dd None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:49.127170 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeee02b-a8c7-49f1-8322-af17231be6dd None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:49.127430 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeee02b-a8c7-49f1-8322-af17231be6dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:49.178878 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78c8d04-70cc-4210-825e-b02d963eb792 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:49.181311 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-b78c8d04-70cc-4210-825e-b02d963eb792 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:49.195429 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-4eeee02b-a8c7-49f1-8322-af17231be6dd None None] MFA Rules not processed for user `64591b8931e84f85b12d587c2ff19ce8`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:49.197129 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-4eeee02b-a8c7-49f1-8322-af17231be6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:49.388399 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 890/1792] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:49.403148 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeee02b-a8c7-49f1-8322-af17231be6dd None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:49.508104 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78c8d04-70cc-4210-825e-b02d963eb792 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:49.532846 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a88fa-bba3-481c-9c34-cc8869050aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:49.534309 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a88fa-bba3-481c-9c34-cc8869050aa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:49.534627 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a88fa-bba3-481c-9c34-cc8869050aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:49.534934 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a88fa-bba3-481c-9c34-cc8869050aa8 admin admin] PATH_INFO: `/v3/users/d7f03f8a45164e9eb5f61b2520e35694` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:49.539292 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a88fa-bba3-481c-9c34-cc8869050aa8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7f03f8a45164e9eb5f61b2520e35694)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:49.542727 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeee02b-a8c7-49f1-8322-af17231be6dd None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:49.547398 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a88fa-bba3-481c-9c34-cc8869050aa8 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:49.551149 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d4c2f-e807-4d6f-8b04-2c4847ca3256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:49.552712 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d4c2f-e807-4d6f-8b04-2c4847ca3256 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:49.553537 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d4c2f-e807-4d6f-8b04-2c4847ca3256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:49.553636 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d4c2f-e807-4d6f-8b04-2c4847ca3256 admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:49.554620 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45d4c2f-e807-4d6f-8b04-2c4847ca3256 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:49.560386 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45d4c2f-e807-4d6f-8b04-2c4847ca3256 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:49.570230 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:49.705926 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 891/1793] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:13:49 2022] POST /identity/v3/projects => generated 375 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:13:49.709513 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d4c2f-e807-4d6f-8b04-2c4847ca3256 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:49.722812 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-df8a88fa-bba3-481c-9c34-cc8869050aa8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7f03f8a45164e9eb5f61b2520e35694'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:49.761606 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-df8a88fa-bba3-481c-9c34-cc8869050aa8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7f03f8a45164e9eb5f61b2520e35694'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:49.781788 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-df8a88fa-bba3-481c-9c34-cc8869050aa8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7f03f8a45164e9eb5f61b2520e35694'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:49.799942 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-df8a88fa-bba3-481c-9c34-cc8869050aa8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7f03f8a45164e9eb5f61b2520e35694'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:49.800421 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-df8a88fa-bba3-481c-9c34-cc8869050aa8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7f03f8a45164e9eb5f61b2520e35694'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:49.814681 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 903/1794] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:13:49 2022] DELETE /identity/v3/users/d7f03f8a45164e9eb5f61b2520e35694 => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:49.816359 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e662d7b-a2c5-48e3-bd12-bc366ce9e7fb None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:49.816649 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e662d7b-a2c5-48e3-bd12-bc366ce9e7fb None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:49.816928 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e662d7b-a2c5-48e3-bd12-bc366ce9e7fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:49.841105 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=85,keystone:INSERT=4,keystone:DELETE=11 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:49.891401 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-1e662d7b-a2c5-48e3-bd12-bc366ce9e7fb None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:49.892791 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-1e662d7b-a2c5-48e3-bd12-bc366ce9e7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:49.912739 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d4c2f-e807-4d6f-8b04-2c4847ca3256 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:50.097869 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f03a6ef-f520-4986-b54a-452f6270481c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:50.101331 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f03a6ef-f520-4986-b54a-452f6270481c admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:50.102649 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f03a6ef-f520-4986-b54a-452f6270481c admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:50.102945 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f03a6ef-f520-4986-b54a-452f6270481c admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:50.105158 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f03a6ef-f520-4986-b54a-452f6270481c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:50.109466 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f03a6ef-f520-4986-b54a-452f6270481c admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:50.171413 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 904/1795] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:13:50.173027 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 892/1796] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:13:49 2022] POST /identity/v3/users => generated 294 bytes in 466 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:13:50.174192 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e662d7b-a2c5-48e3-bd12-bc366ce9e7fb None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:50.176794 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de8acf-87ce-4bb2-807f-6ea5bf4afbd3 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:50.177038 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de8acf-87ce-4bb2-807f-6ea5bf4afbd3 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:50.177281 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de8acf-87ce-4bb2-807f-6ea5bf4afbd3 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:50.178628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-1e662d7b-a2c5-48e3-bd12-bc366ce9e7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:50.185720 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 893/1797] 10.222.0.42 () {54 vars in 921 bytes} [Thu Sep 8 01:13:50 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:13:50.237475 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e662d7b-a2c5-48e3-bd12-bc366ce9e7fb None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:50.254514 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5249504-c2b6-46ff-9448-6f1371566f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:50.255330 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5249504-c2b6-46ff-9448-6f1371566f63 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:50.255791 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5249504-c2b6-46ff-9448-6f1371566f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:50.256083 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5249504-c2b6-46ff-9448-6f1371566f63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:50.256863 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5249504-c2b6-46ff-9448-6f1371566f63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:50.259537 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5249504-c2b6-46ff-9448-6f1371566f63 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:50.263566 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a32f153-bdc6-4a31-8330-30cdd7b173e9 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:50.264210 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a32f153-bdc6-4a31-8330-30cdd7b173e9 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:50.264746 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a32f153-bdc6-4a31-8330-30cdd7b173e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:50.327133 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-9a32f153-bdc6-4a31-8330-30cdd7b173e9 None None] MFA Rules not processed for user `ae643cfdcb164aa9b474747cd21aa116`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:50.330442 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-9a32f153-bdc6-4a31-8330-30cdd7b173e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:50.331467 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 905/1798] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:13:50.334150 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5249504-c2b6-46ff-9448-6f1371566f63 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:50.348179 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5249504-c2b6-46ff-9448-6f1371566f63 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:50.354312 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330bf2c9-84b8-4f7a-8c9c-27645a127158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:50.355163 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330bf2c9-84b8-4f7a-8c9c-27645a127158 admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:50.355447 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330bf2c9-84b8-4f7a-8c9c-27645a127158 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:50.355798 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330bf2c9-84b8-4f7a-8c9c-27645a127158 admin admin] PATH_INFO: `/v3/projects/1d2ae68d2860489fb83d7955360fd474/users/54269db2e51a460fb76db4ff601ef9a8/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:50.374574 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330bf2c9-84b8-4f7a-8c9c-27645a127158 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d2ae68d2860489fb83d7955360fd474, user_id=54269db2e51a460fb76db4ff601ef9a8, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:50.377751 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330bf2c9-84b8-4f7a-8c9c-27645a127158 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:50.411794 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 906/1799] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:13:50 2022] PUT /identity/v3/projects/1d2ae68d2860489fb83d7955360fd474/users/54269db2e51a460fb76db4ff601ef9a8/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:50.413112 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330bf2c9-84b8-4f7a-8c9c-27645a127158 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:50.419848 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-330bf2c9-84b8-4f7a-8c9c-27645a127158 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:50.593282 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 894/1800] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Sep 8 01:13:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 08 01:13:50.594738 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a32f153-bdc6-4a31-8330-30cdd7b173e9 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:50.616203 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330bf2c9-84b8-4f7a-8c9c-27645a127158 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:50.632185 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a211df8a-3128-4748-b37f-7fe6bd44f68c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:50.632185 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211df8a-3128-4748-b37f-7fe6bd44f68c service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:50.632185 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211df8a-3128-4748-b37f-7fe6bd44f68c service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:50.632789 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211df8a-3128-4748-b37f-7fe6bd44f68c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:50.646157 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a32f153-bdc6-4a31-8330-30cdd7b173e9 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:50.666920 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:50.889407 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a211df8a-3128-4748-b37f-7fe6bd44f68c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:50.901880 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a211df8a-3128-4748-b37f-7fe6bd44f68c service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:50.936978 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 907/1801] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:13:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:13:50.938323 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e794ff-14e7-4bb0-b66b-6cfddd4d6210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:50.940108 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a211df8a-3128-4748-b37f-7fe6bd44f68c service neutron] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:50.944175 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-a211df8a-3128-4748-b37f-7fe6bd44f68c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:50.948346 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e794ff-14e7-4bb0-b66b-6cfddd4d6210 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:50.952817 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e794ff-14e7-4bb0-b66b-6cfddd4d6210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:50.953350 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e794ff-14e7-4bb0-b66b-6cfddd4d6210 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:50.954058 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e794ff-14e7-4bb0-b66b-6cfddd4d6210 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:50.955851 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e794ff-14e7-4bb0-b66b-6cfddd4d6210 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:51.004712 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 895/1802] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:13:51.005815 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e794ff-14e7-4bb0-b66b-6cfddd4d6210 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:51.025988 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:51.088044 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e794ff-14e7-4bb0-b66b-6cfddd4d6210 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:51.132028 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d266a38-4c3a-4aae-896d-14cd070fae4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:51.135611 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d266a38-4c3a-4aae-896d-14cd070fae4f admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:51.137147 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d266a38-4c3a-4aae-896d-14cd070fae4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:51.138135 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d266a38-4c3a-4aae-896d-14cd070fae4f admin admin] PATH_INFO: `/v3/projects/1d2ae68d2860489fb83d7955360fd474/users/54269db2e51a460fb76db4ff601ef9a8/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:51.153395 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d266a38-4c3a-4aae-896d-14cd070fae4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d2ae68d2860489fb83d7955360fd474, user_id=54269db2e51a460fb76db4ff601ef9a8, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:51.160025 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d266a38-4c3a-4aae-896d-14cd070fae4f admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:51.198759 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 896/1803] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:13:51 2022] PUT /identity/v3/projects/1d2ae68d2860489fb83d7955360fd474/users/54269db2e51a460fb76db4ff601ef9a8/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:51.215209 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f176b44-c0d7-477f-abe5-967edff8e902 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:51.215940 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f176b44-c0d7-477f-abe5-967edff8e902 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:51.216627 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f176b44-c0d7-477f-abe5-967edff8e902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:51.230910 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a211df8a-3128-4748-b37f-7fe6bd44f68c service neutron] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:51.388040 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-7f176b44-c0d7-477f-abe5-967edff8e902 None None] MFA Rules not processed for user `54269db2e51a460fb76db4ff601ef9a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:51.389659 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-7f176b44-c0d7-477f-abe5-967edff8e902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:51.501936 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae9abd0-f900-4472-a578-74a6cd9ab68c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae643cfdcb164aa9b474747cd21aa116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:51.504615 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae9abd0-f900-4472-a578-74a6cd9ab68c service nova] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:51.509566 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae9abd0-f900-4472-a578-74a6cd9ab68c service nova] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:51.509986 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae9abd0-f900-4472-a578-74a6cd9ab68c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:51.572708 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae9abd0-f900-4472-a578-74a6cd9ab68c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:51.583453 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae9abd0-f900-4472-a578-74a6cd9ab68c service nova] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:51.671365 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 908/1804] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:13:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 732 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:13:51.737888 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 897/1805] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:51 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 526 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:13:51.764639 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae9abd0-f900-4472-a578-74a6cd9ab68c service nova] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:51.766999 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-6ae9abd0-f900-4472-a578-74a6cd9ab68c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:51.951717 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae9abd0-f900-4472-a578-74a6cd9ab68c service nova] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:51.963049 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b0d415-4e4b-409d-8616-f528b79332aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:51.963523 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b0d415-4e4b-409d-8616-f528b79332aa service cinder] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:51.963815 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b0d415-4e4b-409d-8616-f528b79332aa service cinder] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:51.964175 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b0d415-4e4b-409d-8616-f528b79332aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:52.004931 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b0d415-4e4b-409d-8616-f528b79332aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:52.006579 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b0d415-4e4b-409d-8616-f528b79332aa service cinder] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:52.053195 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 909/1806] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:13:51 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:13:52.252410 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f176b44-c0d7-477f-abe5-967edff8e902 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:52.267205 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f176b44-c0d7-477f-abe5-967edff8e902 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:52.275950 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbbe1af-e3b5-465b-8d13-f09712e65be3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:52.277763 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbbe1af-e3b5-465b-8d13-f09712e65be3 service cinder] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:52.278956 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbbe1af-e3b5-465b-8d13-f09712e65be3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:52.279323 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbbe1af-e3b5-465b-8d13-f09712e65be3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:52.297805 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-5dbbe1af-e3b5-465b-8d13-f09712e65be3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:52.442205 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbbe1af-e3b5-465b-8d13-f09712e65be3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:52.449077 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbbe1af-e3b5-465b-8d13-f09712e65be3 service cinder] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:52.517749 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 898/1807] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:13:52 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:13:53.057176 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b0d415-4e4b-409d-8616-f528b79332aa service cinder] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:53.058859 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-91b0d415-4e4b-409d-8616-f528b79332aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:53.065734 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0173b-0099-4e56-99c8-8b86b056a04a None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:53.065734 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0173b-0099-4e56-99c8-8b86b056a04a None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:53.066219 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0173b-0099-4e56-99c8-8b86b056a04a None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:53.068735 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 899/1808] 10.222.0.42 () {54 vars in 921 bytes} [Thu Sep 8 01:13:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:13:53.089833 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c0173b-0099-4e56-99c8-8b86b056a04a None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:53.164220 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:53.249679 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:53.275737 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c0173b-0099-4e56-99c8-8b86b056a04a None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:53.295784 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e38651-2b73-4d48-9d98-a704b8dfd80c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:53.303611 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e38651-2b73-4d48-9d98-a704b8dfd80c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:53.303939 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e38651-2b73-4d48-9d98-a704b8dfd80c admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:53.307002 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e38651-2b73-4d48-9d98-a704b8dfd80c admin admin] PATH_INFO: `/v3/users/64591b8931e84f85b12d587c2ff19ce8` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:53.312306 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e38651-2b73-4d48-9d98-a704b8dfd80c admin admin] RBAC: Authorizing `identity:delete_user(user_id=64591b8931e84f85b12d587c2ff19ce8)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:53.316709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e38651-2b73-4d48-9d98-a704b8dfd80c admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:53.356448 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b0d415-4e4b-409d-8616-f528b79332aa service cinder] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:53.380047 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05da03e3-6c19-4912-88a7-b6e8a2502b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:53.380846 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da03e3-6c19-4912-88a7-b6e8a2502b5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:53.385704 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da03e3-6c19-4912-88a7-b6e8a2502b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:53.386086 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da03e3-6c19-4912-88a7-b6e8a2502b5e admin admin] PATH_INFO: `/v3/projects/ce6f9e2e136c4ea4b76c77acd5a7b928` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:53.390279 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05da03e3-6c19-4912-88a7-b6e8a2502b5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce6f9e2e136c4ea4b76c77acd5a7b928)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:53.393108 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05da03e3-6c19-4912-88a7-b6e8a2502b5e admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:53.512185 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-27e38651-2b73-4d48-9d98-a704b8dfd80c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64591b8931e84f85b12d587c2ff19ce8'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:53.566930 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-05da03e3-6c19-4912-88a7-b6e8a2502b5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce6f9e2e136c4ea4b76c77acd5a7b928', 'request_id': 'req-05da03e3-6c19-4912-88a7-b6e8a2502b5e'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:53.576269 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-27e38651-2b73-4d48-9d98-a704b8dfd80c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64591b8931e84f85b12d587c2ff19ce8'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:53.582270 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-05da03e3-6c19-4912-88a7-b6e8a2502b5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce6f9e2e136c4ea4b76c77acd5a7b928', 'request_id': 'req-05da03e3-6c19-4912-88a7-b6e8a2502b5e'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:53.598717 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-05da03e3-6c19-4912-88a7-b6e8a2502b5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce6f9e2e136c4ea4b76c77acd5a7b928', 'request_id': 'req-05da03e3-6c19-4912-88a7-b6e8a2502b5e'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:53.632802 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-27e38651-2b73-4d48-9d98-a704b8dfd80c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64591b8931e84f85b12d587c2ff19ce8'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:53.643010 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-27e38651-2b73-4d48-9d98-a704b8dfd80c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64591b8931e84f85b12d587c2ff19ce8'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:53.648991 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-05da03e3-6c19-4912-88a7-b6e8a2502b5e admin admin] The token cache is being invalidate because project ce6f9e2e136c4ea4b76c77acd5a7b928 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:13:53.650778 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-05da03e3-6c19-4912-88a7-b6e8a2502b5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:53.653549 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-05da03e3-6c19-4912-88a7-b6e8a2502b5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:53.657878 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 910/1809] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:13:53 2022] DELETE /identity/v3/projects/ce6f9e2e136c4ea4b76c77acd5a7b928 => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:53.659393 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c5220-0be1-4124-b5d5-6b409ddf31d9 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:53.664021 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c5220-0be1-4124-b5d5-6b409ddf31d9 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:53.664627 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c5220-0be1-4124-b5d5-6b409ddf31d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:53.679508 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-27e38651-2b73-4d48-9d98-a704b8dfd80c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64591b8931e84f85b12d587c2ff19ce8'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:53.701163 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 900/1810] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:13:53 2022] DELETE /identity/v3/users/64591b8931e84f85b12d587c2ff19ce8 => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:13:53.706088 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e38651-2b73-4d48-9d98-a704b8dfd80c admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:53.711247 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-27e38651-2b73-4d48-9d98-a704b8dfd80c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:53.718077 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-325c5220-0be1-4124-b5d5-6b409ddf31d9 None None] MFA Rules not processed for user `ae643cfdcb164aa9b474747cd21aa116`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:53.721699 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-325c5220-0be1-4124-b5d5-6b409ddf31d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:53.927445 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e38651-2b73-4d48-9d98-a704b8dfd80c admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:53.941244 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 911/1811] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Sep 8 01:13:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:53.942484 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9f03c1-85d0-49e7-adb7-8790560ed363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:53.943147 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f03c1-85d0-49e7-adb7-8790560ed363 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:53.943428 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f03c1-85d0-49e7-adb7-8790560ed363 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:53.943707 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f03c1-85d0-49e7-adb7-8790560ed363 admin admin] PATH_INFO: `/v3/users/85c96ff95f92435d8df7a9950be3e076` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:53.944008 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d769f-0425-4efb-8167-2d110beb4ae9 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:53.944459 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d769f-0425-4efb-8167-2d110beb4ae9 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:53.944875 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d769f-0425-4efb-8167-2d110beb4ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:53.945883 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9f03c1-85d0-49e7-adb7-8790560ed363 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85c96ff95f92435d8df7a9950be3e076)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:53.952898 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9f03c1-85d0-49e7-adb7-8790560ed363 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:53.995616 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-5c8d769f-0425-4efb-8167-2d110beb4ae9 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:53.997042 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-5c8d769f-0425-4efb-8167-2d110beb4ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:54.009003 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 912/1812] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:54.010586 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8d769f-0425-4efb-8167-2d110beb4ae9 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:54.013979 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-5c8d769f-0425-4efb-8167-2d110beb4ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:54.040484 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8d769f-0425-4efb-8167-2d110beb4ae9 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:54.049658 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd621dcb-ab09-49b0-9f5e-fd1996f64ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:54.050871 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd621dcb-ab09-49b0-9f5e-fd1996f64ade admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:54.050871 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd621dcb-ab09-49b0-9f5e-fd1996f64ade admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:54.050871 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd621dcb-ab09-49b0-9f5e-fd1996f64ade admin admin] PATH_INFO: `/v3/users/a483fa35bddf4ceab6725166c2fc87d7` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:54.052623 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd621dcb-ab09-49b0-9f5e-fd1996f64ade admin admin] RBAC: Authorizing `identity:delete_user(user_id=a483fa35bddf4ceab6725166c2fc87d7)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:54.055225 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd621dcb-ab09-49b0-9f5e-fd1996f64ade admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:54.061154 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:54.070884 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-6d9f03c1-85d0-49e7-adb7-8790560ed363 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85c96ff95f92435d8df7a9950be3e076'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:54.131158 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-6d9f03c1-85d0-49e7-adb7-8790560ed363 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85c96ff95f92435d8df7a9950be3e076'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:54.186091 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-6d9f03c1-85d0-49e7-adb7-8790560ed363 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85c96ff95f92435d8df7a9950be3e076'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:54.187752 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-6d9f03c1-85d0-49e7-adb7-8790560ed363 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85c96ff95f92435d8df7a9950be3e076'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:54.192003 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-dd621dcb-ab09-49b0-9f5e-fd1996f64ade admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a483fa35bddf4ceab6725166c2fc87d7'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:54.219834 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-6d9f03c1-85d0-49e7-adb7-8790560ed363 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85c96ff95f92435d8df7a9950be3e076'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:54.242882 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:54.252961 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 901/1813] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:13:53 2022] DELETE /identity/v3/users/85c96ff95f92435d8df7a9950be3e076 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:54.252961 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9f03c1-85d0-49e7-adb7-8790560ed363 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:54.259426 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-6d9f03c1-85d0-49e7-adb7-8790560ed363 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:54.270299 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-dd621dcb-ab09-49b0-9f5e-fd1996f64ade admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a483fa35bddf4ceab6725166c2fc87d7'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:54.321581 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-dd621dcb-ab09-49b0-9f5e-fd1996f64ade admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a483fa35bddf4ceab6725166c2fc87d7'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:54.339816 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-dd621dcb-ab09-49b0-9f5e-fd1996f64ade admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a483fa35bddf4ceab6725166c2fc87d7'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:54.340869 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-dd621dcb-ab09-49b0-9f5e-fd1996f64ade admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a483fa35bddf4ceab6725166c2fc87d7'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:54.356778 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 913/1814] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:13:54 2022] DELETE /identity/v3/users/a483fa35bddf4ceab6725166c2fc87d7 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:54.361717 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0de7cf-4392-4a57-8ca7-c8b4da4c73d0 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:54.361996 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0de7cf-4392-4a57-8ca7-c8b4da4c73d0 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:54.362677 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0de7cf-4392-4a57-8ca7-c8b4da4c73d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:54.403912 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-6c0de7cf-4392-4a57-8ca7-c8b4da4c73d0 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:54.405176 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-6c0de7cf-4392-4a57-8ca7-c8b4da4c73d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:54.459916 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9f03c1-85d0-49e7-adb7-8790560ed363 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:54.578274 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 914/1815] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:13:54.651469 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6828eec8-1087-47ab-b25f-5120f7897231 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:54.652250 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828eec8-1087-47ab-b25f-5120f7897231 service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:54.652613 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828eec8-1087-47ab-b25f-5120f7897231 service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:54.653295 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828eec8-1087-47ab-b25f-5120f7897231 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:54.660162 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-6828eec8-1087-47ab-b25f-5120f7897231 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:54.698595 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6828eec8-1087-47ab-b25f-5120f7897231 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:54.700285 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6828eec8-1087-47ab-b25f-5120f7897231 service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:54.724763 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 902/1816] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:13:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:13:54.763416 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0de7cf-4392-4a57-8ca7-c8b4da4c73d0 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:54.765260 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-6c0de7cf-4392-4a57-8ca7-c8b4da4c73d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:54.780835 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0de7cf-4392-4a57-8ca7-c8b4da4c73d0 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:54.785018 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bd089a-b89a-47e2-8e34-003c42d952d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:54.785949 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bd089a-b89a-47e2-8e34-003c42d952d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:54.786386 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bd089a-b89a-47e2-8e34-003c42d952d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:54.786803 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bd089a-b89a-47e2-8e34-003c42d952d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:54.800744 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bd089a-b89a-47e2-8e34-003c42d952d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:54.803682 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bd089a-b89a-47e2-8e34-003c42d952d1 service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:54.819835 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 915/1817] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:13:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:13:54.834690 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6828eec8-1087-47ab-b25f-5120f7897231 service neutron] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:54.836899 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-6828eec8-1087-47ab-b25f-5120f7897231 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:55.031022 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6828eec8-1087-47ab-b25f-5120f7897231 service neutron] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:55.048311 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb3a1b6-a318-4bd9-b153-7c4d39fde853 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae643cfdcb164aa9b474747cd21aa116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:55.049067 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb3a1b6-a318-4bd9-b153-7c4d39fde853 service nova] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:55.049381 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb3a1b6-a318-4bd9-b153-7c4d39fde853 service nova] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:55.049799 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb3a1b6-a318-4bd9-b153-7c4d39fde853 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:55.052155 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-7eb3a1b6-a318-4bd9-b153-7c4d39fde853 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:55.082562 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb3a1b6-a318-4bd9-b153-7c4d39fde853 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:55.085093 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb3a1b6-a318-4bd9-b153-7c4d39fde853 service nova] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:55.109225 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 903/1818] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:13:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:13:56.725036 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bd089a-b89a-47e2-8e34-003c42d952d1 service neutron] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:56.727749 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-93bd089a-b89a-47e2-8e34-003c42d952d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:56.781880 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bd089a-b89a-47e2-8e34-003c42d952d1 service neutron] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:56.799263 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163e9eca-149c-47cd-9e92-c4c24065b665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:56.800012 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e9eca-149c-47cd-9e92-c4c24065b665 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:56.800421 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e9eca-149c-47cd-9e92-c4c24065b665 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:56.800720 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e9eca-149c-47cd-9e92-c4c24065b665 admin admin] PATH_INFO: `/v3/projects/39c5a342a8084982a835660cd17ae080` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:56.804992 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163e9eca-149c-47cd-9e92-c4c24065b665 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39c5a342a8084982a835660cd17ae080)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:56.808167 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163e9eca-149c-47cd-9e92-c4c24065b665 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:57.008349 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-163e9eca-149c-47cd-9e92-c4c24065b665 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39c5a342a8084982a835660cd17ae080', 'request_id': 'req-163e9eca-149c-47cd-9e92-c4c24065b665'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:57.031965 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-163e9eca-149c-47cd-9e92-c4c24065b665 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39c5a342a8084982a835660cd17ae080', 'request_id': 'req-163e9eca-149c-47cd-9e92-c4c24065b665'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:57.059605 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-163e9eca-149c-47cd-9e92-c4c24065b665 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39c5a342a8084982a835660cd17ae080', 'request_id': 'req-163e9eca-149c-47cd-9e92-c4c24065b665'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:57.083340 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb3a1b6-a318-4bd9-b153-7c4d39fde853 service nova] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:57.084982 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-7eb3a1b6-a318-4bd9-b153-7c4d39fde853 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:57.098836 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-163e9eca-149c-47cd-9e92-c4c24065b665 admin admin] The token cache is being invalidate because project 39c5a342a8084982a835660cd17ae080 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:13:57.099210 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-163e9eca-149c-47cd-9e92-c4c24065b665 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:57.099890 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-163e9eca-149c-47cd-9e92-c4c24065b665 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:57.102401 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:57.103123 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 916/1819] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:13:56 2022] DELETE /identity/v3/projects/39c5a342a8084982a835660cd17ae080 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:13:57.106598 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163e9eca-149c-47cd-9e92-c4c24065b665 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:57.111270 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-163e9eca-149c-47cd-9e92-c4c24065b665 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:57.121819 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb3a1b6-a318-4bd9-b153-7c4d39fde853 service nova] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:57.223607 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:57.278656 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163e9eca-149c-47cd-9e92-c4c24065b665 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:57.304353 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93da28ee-4a58-454f-9bad-2b15882b065f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:57.305017 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93da28ee-4a58-454f-9bad-2b15882b065f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:57.305449 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93da28ee-4a58-454f-9bad-2b15882b065f admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:57.305791 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93da28ee-4a58-454f-9bad-2b15882b065f admin admin] PATH_INFO: `/v3/users/54269db2e51a460fb76db4ff601ef9a8` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:57.307433 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93da28ee-4a58-454f-9bad-2b15882b065f admin admin] RBAC: Authorizing `identity:delete_user(user_id=54269db2e51a460fb76db4ff601ef9a8)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:57.310749 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93da28ee-4a58-454f-9bad-2b15882b065f admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:57.329038 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d2550b-578c-4c42-8098-7b5082f9ce99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:57.330157 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d2550b-578c-4c42-8098-7b5082f9ce99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:57.330609 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d2550b-578c-4c42-8098-7b5082f9ce99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:57.330884 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d2550b-578c-4c42-8098-7b5082f9ce99 admin admin] PATH_INFO: `/v3/projects/46a33e5bf04848f78af61abe424cf3f1` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:57.338605 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d2550b-578c-4c42-8098-7b5082f9ce99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46a33e5bf04848f78af61abe424cf3f1)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:57.340923 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d2550b-578c-4c42-8098-7b5082f9ce99 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:57.415901 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-93da28ee-4a58-454f-9bad-2b15882b065f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54269db2e51a460fb76db4ff601ef9a8'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:57.433413 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-93da28ee-4a58-454f-9bad-2b15882b065f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54269db2e51a460fb76db4ff601ef9a8'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:57.441734 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-93da28ee-4a58-454f-9bad-2b15882b065f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54269db2e51a460fb76db4ff601ef9a8'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:57.456552 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-93da28ee-4a58-454f-9bad-2b15882b065f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54269db2e51a460fb76db4ff601ef9a8'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:57.458443 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-52d2550b-578c-4c42-8098-7b5082f9ce99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46a33e5bf04848f78af61abe424cf3f1', 'request_id': 'req-52d2550b-578c-4c42-8098-7b5082f9ce99'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:57.459286 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-93da28ee-4a58-454f-9bad-2b15882b065f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54269db2e51a460fb76db4ff601ef9a8'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:57.474087 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 917/1820] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:13:57 2022] DELETE /identity/v3/users/54269db2e51a460fb76db4ff601ef9a8 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:13:57.476436 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-52d2550b-578c-4c42-8098-7b5082f9ce99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46a33e5bf04848f78af61abe424cf3f1', 'request_id': 'req-52d2550b-578c-4c42-8098-7b5082f9ce99'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:57.485850 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-52d2550b-578c-4c42-8098-7b5082f9ce99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46a33e5bf04848f78af61abe424cf3f1', 'request_id': 'req-52d2550b-578c-4c42-8098-7b5082f9ce99'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:57.503281 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-52d2550b-578c-4c42-8098-7b5082f9ce99 admin admin] The token cache is being invalidate because project 46a33e5bf04848f78af61abe424cf3f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81661) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:13:57.503712 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-52d2550b-578c-4c42-8098-7b5082f9ce99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:57.504518 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-52d2550b-578c-4c42-8098-7b5082f9ce99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:57.506941 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 904/1821] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:13:57 2022] DELETE /identity/v3/projects/46a33e5bf04848f78af61abe424cf3f1 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:57.513767 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6473c3-ee05-4bc5-8e50-b95ac5143a71 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:57.514342 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6473c3-ee05-4bc5-8e50-b95ac5143a71 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:57.514941 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6473c3-ee05-4bc5-8e50-b95ac5143a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:57.539719 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-ad6473c3-ee05-4bc5-8e50-b95ac5143a71 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:57.540993 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-ad6473c3-ee05-4bc5-8e50-b95ac5143a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:57.616763 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 918/1822] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:57.629182 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67532672-044a-446e-a764-966db8e33b71 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:57.629523 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67532672-044a-446e-a764-966db8e33b71 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:57.629852 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67532672-044a-446e-a764-966db8e33b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:57.690885 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-67532672-044a-446e-a764-966db8e33b71 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:57.692379 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-67532672-044a-446e-a764-966db8e33b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:57.699475 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6473c3-ee05-4bc5-8e50-b95ac5143a71 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:57.701091 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-ad6473c3-ee05-4bc5-8e50-b95ac5143a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:57.726035 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 905/1823] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:57.728473 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6473c3-ee05-4bc5-8e50-b95ac5143a71 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:57.732857 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67532672-044a-446e-a764-966db8e33b71 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:57.744689 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f1fe5-baa2-4f81-b047-f48b9710dcb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:57.747185 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f1fe5-baa2-4f81-b047-f48b9710dcb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:57.747459 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67532672-044a-446e-a764-966db8e33b71 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:57.749135 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f1fe5-baa2-4f81-b047-f48b9710dcb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:57.749639 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f1fe5-baa2-4f81-b047-f48b9710dcb8 admin admin] PATH_INFO: `/v3/projects/861dc116bd6b420f942982c507e6fb14` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:57.753702 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5f1fe5-baa2-4f81-b047-f48b9710dcb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=861dc116bd6b420f942982c507e6fb14)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:57.753928 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42271940-e6d0-4c5e-a88a-aed9020c4715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:57.754656 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42271940-e6d0-4c5e-a88a-aed9020c4715 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:57.754972 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42271940-e6d0-4c5e-a88a-aed9020c4715 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:57.755216 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42271940-e6d0-4c5e-a88a-aed9020c4715 admin admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:57.755975 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42271940-e6d0-4c5e-a88a-aed9020c4715 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:13:57.756447 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42271940-e6d0-4c5e-a88a-aed9020c4715 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:57.758073 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42271940-e6d0-4c5e-a88a-aed9020c4715 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:57.759680 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5f1fe5-baa2-4f81-b047-f48b9710dcb8 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:57.781023 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 906/1824] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:13:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:13:57.800252 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42271940-e6d0-4c5e-a88a-aed9020c4715 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:57.834481 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42271940-e6d0-4c5e-a88a-aed9020c4715 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:57.856190 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-0a5f1fe5-baa2-4f81-b047-f48b9710dcb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '861dc116bd6b420f942982c507e6fb14', 'request_id': 'req-0a5f1fe5-baa2-4f81-b047-f48b9710dcb8'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:57.865203 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-0a5f1fe5-baa2-4f81-b047-f48b9710dcb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '861dc116bd6b420f942982c507e6fb14', 'request_id': 'req-0a5f1fe5-baa2-4f81-b047-f48b9710dcb8'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:57.872502 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:57.876234 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-0a5f1fe5-baa2-4f81-b047-f48b9710dcb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '861dc116bd6b420f942982c507e6fb14', 'request_id': 'req-0a5f1fe5-baa2-4f81-b047-f48b9710dcb8'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:57.903197 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-0a5f1fe5-baa2-4f81-b047-f48b9710dcb8 admin admin] The token cache is being invalidate because project 861dc116bd6b420f942982c507e6fb14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:13:57.903702 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-0a5f1fe5-baa2-4f81-b047-f48b9710dcb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:57.905714 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-0a5f1fe5-baa2-4f81-b047-f48b9710dcb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:57.912856 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 919/1825] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:13:57 2022] DELETE /identity/v3/projects/861dc116bd6b420f942982c507e6fb14 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:57.919213 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f1fe5-baa2-4f81-b047-f48b9710dcb8 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:57.920070 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d76e53-b52d-44d6-9d96-d87c175aecfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:57.920823 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d76e53-b52d-44d6-9d96-d87c175aecfe admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:57.921174 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d76e53-b52d-44d6-9d96-d87c175aecfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:57.921301 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-0a5f1fe5-baa2-4f81-b047-f48b9710dcb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:57.922678 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d76e53-b52d-44d6-9d96-d87c175aecfe admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:57.923449 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d76e53-b52d-44d6-9d96-d87c175aecfe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:57.926628 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d76e53-b52d-44d6-9d96-d87c175aecfe admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:57.952103 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f1fe5-baa2-4f81-b047-f48b9710dcb8 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:57.955446 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:57.980262 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 907/1826] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:13:57 2022] POST /identity/v3/projects => generated 383 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:13:57.981427 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d76e53-b52d-44d6-9d96-d87c175aecfe admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:57.986992 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-03d76e53-b52d-44d6-9d96-d87c175aecfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:57.990782 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca32ba87-cbde-476c-9f07-2caff84e2a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:57.995436 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca32ba87-cbde-476c-9f07-2caff84e2a9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:57.996810 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca32ba87-cbde-476c-9f07-2caff84e2a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:58.000481 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca32ba87-cbde-476c-9f07-2caff84e2a9e admin admin] PATH_INFO: `/v3/users/5510182c76f1481a85bbc3dde966ce7d` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:58.008113 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca32ba87-cbde-476c-9f07-2caff84e2a9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5510182c76f1481a85bbc3dde966ce7d)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:58.014417 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca32ba87-cbde-476c-9f07-2caff84e2a9e admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:58.128346 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d76e53-b52d-44d6-9d96-d87c175aecfe admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:58.137055 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d47e6-3cc2-4f08-bd1b-fca710cfcc48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:58.137854 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d47e6-3cc2-4f08-bd1b-fca710cfcc48 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:58.138203 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d47e6-3cc2-4f08-bd1b-fca710cfcc48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:58.138590 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d47e6-3cc2-4f08-bd1b-fca710cfcc48 admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:58.139301 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767d47e6-3cc2-4f08-bd1b-fca710cfcc48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:58.142912 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767d47e6-3cc2-4f08-bd1b-fca710cfcc48 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:58.161177 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-ca32ba87-cbde-476c-9f07-2caff84e2a9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5510182c76f1481a85bbc3dde966ce7d'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:58.179648 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 908/1827] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:13:57 2022] POST /identity/v3/users => generated 292 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 01:13:58.183776 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d47e6-3cc2-4f08-bd1b-fca710cfcc48 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:58.200539 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d47e6-3cc2-4f08-bd1b-fca710cfcc48 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:58.206280 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-ca32ba87-cbde-476c-9f07-2caff84e2a9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5510182c76f1481a85bbc3dde966ce7d'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:58.208470 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a5a5f8-9e87-4163-b312-7dcb7695368d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:58.209174 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5a5f8-9e87-4163-b312-7dcb7695368d admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:58.209518 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5a5f8-9e87-4163-b312-7dcb7695368d admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:58.209857 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5a5f8-9e87-4163-b312-7dcb7695368d admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:58.210685 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a5a5f8-9e87-4163-b312-7dcb7695368d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:58.212657 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a5a5f8-9e87-4163-b312-7dcb7695368d admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:58.218230 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-ca32ba87-cbde-476c-9f07-2caff84e2a9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5510182c76f1481a85bbc3dde966ce7d'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:58.229257 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-ca32ba87-cbde-476c-9f07-2caff84e2a9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5510182c76f1481a85bbc3dde966ce7d'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:58.230309 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-ca32ba87-cbde-476c-9f07-2caff84e2a9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5510182c76f1481a85bbc3dde966ce7d'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:58.236885 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 909/1828] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:13:58.238091 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a5a5f8-9e87-4163-b312-7dcb7695368d admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:58.239350 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 920/1829] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:13:57 2022] DELETE /identity/v3/users/5510182c76f1481a85bbc3dde966ce7d => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:58.243178 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-28a5a5f8-9e87-4163-b312-7dcb7695368d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:58.258980 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a5a5f8-9e87-4163-b312-7dcb7695368d admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:58.349950 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77877979-8524-495c-9e53-703bbb3e8ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:58.350999 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77877979-8524-495c-9e53-703bbb3e8ccc admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:58.351375 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77877979-8524-495c-9e53-703bbb3e8ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:58.351659 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77877979-8524-495c-9e53-703bbb3e8ccc admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:58.352955 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77877979-8524-495c-9e53-703bbb3e8ccc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:58.354762 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77877979-8524-495c-9e53-703bbb3e8ccc admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:58.382738 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 910/1830] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:13:58.386260 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca32ba87-cbde-476c-9f07-2caff84e2a9e admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:58.402496 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca32ba87-cbde-476c-9f07-2caff84e2a9e admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:58.410455 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77877979-8524-495c-9e53-703bbb3e8ccc admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:58.412609 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-77877979-8524-495c-9e53-703bbb3e8ccc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:58.425031 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3554f80e-b409-4549-8da0-52efef63b362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:58.425579 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554f80e-b409-4549-8da0-52efef63b362 admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:58.425849 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554f80e-b409-4549-8da0-52efef63b362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:58.426137 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554f80e-b409-4549-8da0-52efef63b362 admin admin] PATH_INFO: `/v3/projects/dccb74c6c96b4964a3df57f52a1f8e71/users/8635c569b0bf413aa4eb3937640e40bc/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:58.434663 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77877979-8524-495c-9e53-703bbb3e8ccc admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:58.457698 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6ecb35-f30b-49d9-aeb5-108ee762e0ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:58.458967 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6ecb35-f30b-49d9-aeb5-108ee762e0ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:58.463972 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6ecb35-f30b-49d9-aeb5-108ee762e0ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:58.464993 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3554f80e-b409-4549-8da0-52efef63b362 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dccb74c6c96b4964a3df57f52a1f8e71, user_id=8635c569b0bf413aa4eb3937640e40bc, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:58.465436 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6ecb35-f30b-49d9-aeb5-108ee762e0ef admin admin] PATH_INFO: `/v3/projects/1d2ae68d2860489fb83d7955360fd474` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:58.470293 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6ecb35-f30b-49d9-aeb5-108ee762e0ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d2ae68d2860489fb83d7955360fd474)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:58.474062 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6ecb35-f30b-49d9-aeb5-108ee762e0ef admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:58.476811 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3554f80e-b409-4549-8da0-52efef63b362 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:58.498781 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 921/1831] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:13:58 2022] PUT /identity/v3/projects/dccb74c6c96b4964a3df57f52a1f8e71/users/8635c569b0bf413aa4eb3937640e40bc/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:58.511119 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed36da9-db01-45ad-9ff9-5befc045db36 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:58.511582 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed36da9-db01-45ad-9ff9-5befc045db36 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:58.511949 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed36da9-db01-45ad-9ff9-5befc045db36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:58.557530 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-5ed36da9-db01-45ad-9ff9-5befc045db36 None None] MFA Rules not processed for user `8635c569b0bf413aa4eb3937640e40bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:58.558979 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-5ed36da9-db01-45ad-9ff9-5befc045db36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:58.588339 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-3c6ecb35-f30b-49d9-aeb5-108ee762e0ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d2ae68d2860489fb83d7955360fd474', 'request_id': 'req-3c6ecb35-f30b-49d9-aeb5-108ee762e0ef'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:58.600645 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-3c6ecb35-f30b-49d9-aeb5-108ee762e0ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d2ae68d2860489fb83d7955360fd474', 'request_id': 'req-3c6ecb35-f30b-49d9-aeb5-108ee762e0ef'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:58.609195 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-3c6ecb35-f30b-49d9-aeb5-108ee762e0ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d2ae68d2860489fb83d7955360fd474', 'request_id': 'req-3c6ecb35-f30b-49d9-aeb5-108ee762e0ef'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:58.618664 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:58.628837 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-3c6ecb35-f30b-49d9-aeb5-108ee762e0ef admin admin] The token cache is being invalidate because project 1d2ae68d2860489fb83d7955360fd474 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81661) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:13:58.629403 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-3c6ecb35-f30b-49d9-aeb5-108ee762e0ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:58.631764 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-3c6ecb35-f30b-49d9-aeb5-108ee762e0ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:58.633040 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 911/1832] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:13:58 2022] DELETE /identity/v3/projects/1d2ae68d2860489fb83d7955360fd474 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:58.638081 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85fa12b-4dbf-4338-a8ea-f447d5c5ceb8 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:58.639057 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85fa12b-4dbf-4338-a8ea-f447d5c5ceb8 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:58.639379 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85fa12b-4dbf-4338-a8ea-f447d5c5ceb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:58.663723 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-c85fa12b-4dbf-4338-a8ea-f447d5c5ceb8 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:58.664792 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-c85fa12b-4dbf-4338-a8ea-f447d5c5ceb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:58.673755 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 922/1833] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:58.676843 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed36da9-db01-45ad-9ff9-5befc045db36 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:58.678797 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-5ed36da9-db01-45ad-9ff9-5befc045db36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:58.732620 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:58.753178 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 912/1834] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:58.762185 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645430ea-b563-4f5a-8628-9aad976b4008 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:58.762609 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645430ea-b563-4f5a-8628-9aad976b4008 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:58.763381 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645430ea-b563-4f5a-8628-9aad976b4008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:58.789395 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-645430ea-b563-4f5a-8628-9aad976b4008 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:58.790602 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-645430ea-b563-4f5a-8628-9aad976b4008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:58.806358 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 913/1835] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:58.807414 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645430ea-b563-4f5a-8628-9aad976b4008 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:58.814640 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed36da9-db01-45ad-9ff9-5befc045db36 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:58.822538 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479b2a4d-f517-485b-bc5f-25e114bdd26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:58.823877 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b2a4d-f517-485b-bc5f-25e114bdd26a admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:58.824656 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b2a4d-f517-485b-bc5f-25e114bdd26a admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:58.825151 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b2a4d-f517-485b-bc5f-25e114bdd26a admin admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:58.826376 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479b2a4d-f517-485b-bc5f-25e114bdd26a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:58.829793 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645430ea-b563-4f5a-8628-9aad976b4008 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:58.830605 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479b2a4d-f517-485b-bc5f-25e114bdd26a admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:58.835891 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0ea7aa-db7e-461b-939d-ae98bbcd6908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:58.836596 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ea7aa-db7e-461b-939d-ae98bbcd6908 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:58.836988 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ea7aa-db7e-461b-939d-ae98bbcd6908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:58.837179 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ea7aa-db7e-461b-939d-ae98bbcd6908 admin admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:58.837886 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0ea7aa-db7e-461b-939d-ae98bbcd6908 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:13:58.838212 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0ea7aa-db7e-461b-939d-ae98bbcd6908 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:58.842714 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0ea7aa-db7e-461b-939d-ae98bbcd6908 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:58.870547 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 914/1836] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:13:58.871499 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0ea7aa-db7e-461b-939d-ae98bbcd6908 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:58.900098 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0ea7aa-db7e-461b-939d-ae98bbcd6908 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:58.903467 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 923/1837] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:13:58 2022] POST /identity/v3/projects => generated 383 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:13:58.905129 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479b2a4d-f517-485b-bc5f-25e114bdd26a admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:58.908384 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-479b2a4d-f517-485b-bc5f-25e114bdd26a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:59.039692 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479b2a4d-f517-485b-bc5f-25e114bdd26a admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:59.057177 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96c8061-cbe8-4944-a6fa-1ce1f7f2bd05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:59.059424 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c8061-cbe8-4944-a6fa-1ce1f7f2bd05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:59.060900 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c8061-cbe8-4944-a6fa-1ce1f7f2bd05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:59.061386 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c8061-cbe8-4944-a6fa-1ce1f7f2bd05 admin admin] PATH_INFO: `/v3/projects/209cebd89073488caec6044bfc794fdd` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:59.063190 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f083dd6-534a-47a5-ad62-9b7f2110c156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:59.064269 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f083dd6-534a-47a5-ad62-9b7f2110c156 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:59.064692 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f083dd6-534a-47a5-ad62-9b7f2110c156 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:59.065120 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f083dd6-534a-47a5-ad62-9b7f2110c156 admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:59.066207 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f083dd6-534a-47a5-ad62-9b7f2110c156 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:59.067117 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96c8061-cbe8-4944-a6fa-1ce1f7f2bd05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=209cebd89073488caec6044bfc794fdd)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:59.068641 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f083dd6-534a-47a5-ad62-9b7f2110c156 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:59.072894 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96c8061-cbe8-4944-a6fa-1ce1f7f2bd05 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:59.123186 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 915/1838] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:13:58 2022] POST /identity/v3/projects => generated 385 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:13:59.124797 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f083dd6-534a-47a5-ad62-9b7f2110c156 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:59.180492 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-b96c8061-cbe8-4944-a6fa-1ce1f7f2bd05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '209cebd89073488caec6044bfc794fdd', 'request_id': 'req-b96c8061-cbe8-4944-a6fa-1ce1f7f2bd05'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:59.187943 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-b96c8061-cbe8-4944-a6fa-1ce1f7f2bd05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '209cebd89073488caec6044bfc794fdd', 'request_id': 'req-b96c8061-cbe8-4944-a6fa-1ce1f7f2bd05'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:59.194773 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-b96c8061-cbe8-4944-a6fa-1ce1f7f2bd05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '209cebd89073488caec6044bfc794fdd', 'request_id': 'req-b96c8061-cbe8-4944-a6fa-1ce1f7f2bd05'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:59.209606 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-b96c8061-cbe8-4944-a6fa-1ce1f7f2bd05 admin admin] The token cache is being invalidate because project 209cebd89073488caec6044bfc794fdd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:13:59.210043 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-b96c8061-cbe8-4944-a6fa-1ce1f7f2bd05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:59.210730 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-b96c8061-cbe8-4944-a6fa-1ce1f7f2bd05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:13:59.224120 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 924/1839] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:13:58 2022] DELETE /identity/v3/projects/209cebd89073488caec6044bfc794fdd => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:59.228813 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96c8061-cbe8-4944-a6fa-1ce1f7f2bd05 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:59.232538 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-b96c8061-cbe8-4944-a6fa-1ce1f7f2bd05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:59.273148 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f083dd6-534a-47a5-ad62-9b7f2110c156 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:59.285719 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:59.318041 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:59.363997 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db230ea3-12eb-408f-a0f6-a3971a65cb74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:59.364640 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db230ea3-12eb-408f-a0f6-a3971a65cb74 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:59.364940 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db230ea3-12eb-408f-a0f6-a3971a65cb74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:59.365211 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db230ea3-12eb-408f-a0f6-a3971a65cb74 admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:59.365849 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db230ea3-12eb-408f-a0f6-a3971a65cb74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:59.367781 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db230ea3-12eb-408f-a0f6-a3971a65cb74 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:59.373648 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96c8061-cbe8-4944-a6fa-1ce1f7f2bd05 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:59.388962 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cc63a9-5df5-4ea0-96a4-5a324a81cf9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:59.390055 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cc63a9-5df5-4ea0-96a4-5a324a81cf9f admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:59.390913 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cc63a9-5df5-4ea0-96a4-5a324a81cf9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:59.391272 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cc63a9-5df5-4ea0-96a4-5a324a81cf9f admin admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:59.392237 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cc63a9-5df5-4ea0-96a4-5a324a81cf9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:59.395880 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 916/1840] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:13:59 2022] POST /identity/v3/users => generated 298 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:13:59.395880 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1093adc4-77bc-49a7-a3cf-c7436fc34d49 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:59.395880 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1093adc4-77bc-49a7-a3cf-c7436fc34d49 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:59.395880 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1093adc4-77bc-49a7-a3cf-c7436fc34d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:59.396826 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cc63a9-5df5-4ea0-96a4-5a324a81cf9f admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:59.416958 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-1093adc4-77bc-49a7-a3cf-c7436fc34d49 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:59.418174 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-1093adc4-77bc-49a7-a3cf-c7436fc34d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:59.420031 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 925/1841] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:13:59 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:13:59.420031 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cc63a9-5df5-4ea0-96a4-5a324a81cf9f admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:59.425855 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-d4cc63a9-5df5-4ea0-96a4-5a324a81cf9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:59.431408 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 917/1842] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:59.432318 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1093adc4-77bc-49a7-a3cf-c7436fc34d49 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:59.443873 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1093adc4-77bc-49a7-a3cf-c7436fc34d49 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:59.444263 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cc63a9-5df5-4ea0-96a4-5a324a81cf9f admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:59.448759 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36b0a73-f4a5-4995-97f3-3ecc1bbb6574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:59.449048 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74117e75-9cd0-4b5c-b488-5841a4b1fd2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:59.449348 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b0a73-f4a5-4995-97f3-3ecc1bbb6574 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:59.449623 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b0a73-f4a5-4995-97f3-3ecc1bbb6574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:59.449720 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74117e75-9cd0-4b5c-b488-5841a4b1fd2e admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:59.449891 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b0a73-f4a5-4995-97f3-3ecc1bbb6574 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:59.449984 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74117e75-9cd0-4b5c-b488-5841a4b1fd2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:59.450171 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74117e75-9cd0-4b5c-b488-5841a4b1fd2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:59.450521 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36b0a73-f4a5-4995-97f3-3ecc1bbb6574 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:59.450809 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74117e75-9cd0-4b5c-b488-5841a4b1fd2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:59.452117 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36b0a73-f4a5-4995-97f3-3ecc1bbb6574 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:59.452416 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74117e75-9cd0-4b5c-b488-5841a4b1fd2e admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:59.462662 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 918/1843] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:13:59.463779 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75e779-1172-44d3-b850-ce22c3d48ff5 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:59.464061 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75e779-1172-44d3-b850-ce22c3d48ff5 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:59.464355 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75e779-1172-44d3-b850-ce22c3d48ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:59.468353 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 926/1844] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:13:59.468353 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74117e75-9cd0-4b5c-b488-5841a4b1fd2e admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:59.479092 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74117e75-9cd0-4b5c-b488-5841a4b1fd2e admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:59.483812 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244b1e52-fa34-4938-91af-70c7d2d65e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:59.485040 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b1e52-fa34-4938-91af-70c7d2d65e1a admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:59.485540 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b1e52-fa34-4938-91af-70c7d2d65e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:59.485838 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b1e52-fa34-4938-91af-70c7d2d65e1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:59.486474 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244b1e52-fa34-4938-91af-70c7d2d65e1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:59.486712 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-3b75e779-1172-44d3-b850-ce22c3d48ff5 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:59.487928 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-3b75e779-1172-44d3-b850-ce22c3d48ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:59.488043 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244b1e52-fa34-4938-91af-70c7d2d65e1a admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:59.497209 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 919/1845] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:59.497980 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b75e779-1172-44d3-b850-ce22c3d48ff5 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:59.506367 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 927/1846] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 01:13:59.506367 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244b1e52-fa34-4938-91af-70c7d2d65e1a admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:59.511765 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b75e779-1172-44d3-b850-ce22c3d48ff5 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:59.514304 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244b1e52-fa34-4938-91af-70c7d2d65e1a admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:59.516752 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16888910-b2d4-4e57-9e08-83013a08e282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:59.517412 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16888910-b2d4-4e57-9e08-83013a08e282 admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:59.517694 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16888910-b2d4-4e57-9e08-83013a08e282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:59.517968 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16888910-b2d4-4e57-9e08-83013a08e282 admin admin] PATH_INFO: `/v3/projects/892fe3caf9634fb78a40188f558be562/users/e6aa19872bd3486da5c81caeec578861/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:59.519231 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6759a9a4-266a-4065-b8a6-61f097c5a16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:59.519882 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6759a9a4-266a-4065-b8a6-61f097c5a16d admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:59.520188 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6759a9a4-266a-4065-b8a6-61f097c5a16d admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:59.520506 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6759a9a4-266a-4065-b8a6-61f097c5a16d admin admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:59.521243 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6759a9a4-266a-4065-b8a6-61f097c5a16d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:13:59.521562 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6759a9a4-266a-4065-b8a6-61f097c5a16d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:59.523251 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6759a9a4-266a-4065-b8a6-61f097c5a16d admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:59.533351 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16888910-b2d4-4e57-9e08-83013a08e282 admin admin] RBAC: Authorizing `identity:create_grant(project_id=892fe3caf9634fb78a40188f558be562, user_id=e6aa19872bd3486da5c81caeec578861, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:59.536110 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 928/1847] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:13:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:13:59.536259 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16888910-b2d4-4e57-9e08-83013a08e282 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:59.536747 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6759a9a4-266a-4065-b8a6-61f097c5a16d admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:59.549362 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6759a9a4-266a-4065-b8a6-61f097c5a16d admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:59.552256 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 920/1848] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:13:59 2022] PUT /identity/v3/projects/892fe3caf9634fb78a40188f558be562/users/e6aa19872bd3486da5c81caeec578861/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:59.553286 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16888910-b2d4-4e57-9e08-83013a08e282 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:59.611270 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a464c67a-713b-48c0-ade6-91f0cceded5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:59.612235 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a464c67a-713b-48c0-ade6-91f0cceded5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:59.612670 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a464c67a-713b-48c0-ade6-91f0cceded5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:59.612948 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a464c67a-713b-48c0-ade6-91f0cceded5f admin admin] PATH_INFO: `/v3/projects/dd159c1e58cb42e8ac2f55eaebf2cfd7/users/2f1c9901ebd541d18e1bb3f82ad237c0/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:59.620941 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16888910-b2d4-4e57-9e08-83013a08e282 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:59.626251 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b439bb-1b4e-41c6-a1e0-a36e0ea94cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:59.626945 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b439bb-1b4e-41c6-a1e0-a36e0ea94cfa admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:59.627227 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b439bb-1b4e-41c6-a1e0-a36e0ea94cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:59.627549 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b439bb-1b4e-41c6-a1e0-a36e0ea94cfa admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:59.628384 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b439bb-1b4e-41c6-a1e0-a36e0ea94cfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:59.630634 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b439bb-1b4e-41c6-a1e0-a36e0ea94cfa admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:59.634777 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a464c67a-713b-48c0-ade6-91f0cceded5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd159c1e58cb42e8ac2f55eaebf2cfd7, user_id=2f1c9901ebd541d18e1bb3f82ad237c0, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:59.637693 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a464c67a-713b-48c0-ade6-91f0cceded5f admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:59.653595 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 929/1849] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:13:59 2022] PUT /identity/v3/projects/dd159c1e58cb42e8ac2f55eaebf2cfd7/users/2f1c9901ebd541d18e1bb3f82ad237c0/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:59.660421 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a464c67a-713b-48c0-ade6-91f0cceded5f admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:59.673885 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 921/1850] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:13:59 2022] POST /identity/v3/projects => generated 367 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:13:59.675288 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83059d6-a360-4cf8-afcf-457a75f58ff9 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:59.675577 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83059d6-a360-4cf8-afcf-457a75f58ff9 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:59.675849 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83059d6-a360-4cf8-afcf-457a75f58ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:59.705367 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-d83059d6-a360-4cf8-afcf-457a75f58ff9 None None] MFA Rules not processed for user `2f1c9901ebd541d18e1bb3f82ad237c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:13:59.706561 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-d83059d6-a360-4cf8-afcf-457a75f58ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:13:59.741384 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a464c67a-713b-48c0-ade6-91f0cceded5f admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:59.742615 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:59.767297 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:13:59.791686 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 922/1851] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:13:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:13:59.792698 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83059d6-a360-4cf8-afcf-457a75f58ff9 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:59.815210 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125074e3-a67d-4b56-82ec-d3e062804ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:59.815853 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125074e3-a67d-4b56-82ec-d3e062804ad9 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:59.816148 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125074e3-a67d-4b56-82ec-d3e062804ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:59.816521 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125074e3-a67d-4b56-82ec-d3e062804ad9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:59.817169 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125074e3-a67d-4b56-82ec-d3e062804ad9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:59.818903 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125074e3-a67d-4b56-82ec-d3e062804ad9 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:59.829528 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 930/1852] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:13:59.830315 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125074e3-a67d-4b56-82ec-d3e062804ad9 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:59.840180 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125074e3-a67d-4b56-82ec-d3e062804ad9 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:59.845336 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de91e03-fa25-4d64-ab76-74df4bf69009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:59.846015 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de91e03-fa25-4d64-ab76-74df4bf69009 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:59.846313 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de91e03-fa25-4d64-ab76-74df4bf69009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:59.846595 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de91e03-fa25-4d64-ab76-74df4bf69009 admin admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:59.847308 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de91e03-fa25-4d64-ab76-74df4bf69009 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:59.849423 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de91e03-fa25-4d64-ab76-74df4bf69009 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:59.859164 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83059d6-a360-4cf8-afcf-457a75f58ff9 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:59.864785 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9bc76c-8c31-4571-a86c-f5bee3dd5d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:59.865398 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9bc76c-8c31-4571-a86c-f5bee3dd5d0f admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:59.865678 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9bc76c-8c31-4571-a86c-f5bee3dd5d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:59.865950 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9bc76c-8c31-4571-a86c-f5bee3dd5d0f admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:59.866665 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9bc76c-8c31-4571-a86c-f5bee3dd5d0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:59.869095 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9bc76c-8c31-4571-a86c-f5bee3dd5d0f admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:59.879002 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 931/1853] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:13:59 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:13:59.879690 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de91e03-fa25-4d64-ab76-74df4bf69009 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:59.896048 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 923/1854] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:13:59 2022] POST /identity/v3/users => generated 284 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:13:59.896892 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9bc76c-8c31-4571-a86c-f5bee3dd5d0f admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:59.906638 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9bc76c-8c31-4571-a86c-f5bee3dd5d0f admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:59.949225 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de91e03-fa25-4d64-ab76-74df4bf69009 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:13:59.954719 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12cc1ab-2e89-4d4a-ae7e-ee12f9eaef97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:59.955413 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12cc1ab-2e89-4d4a-ae7e-ee12f9eaef97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:59.955773 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12cc1ab-2e89-4d4a-ae7e-ee12f9eaef97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:59.956115 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12cc1ab-2e89-4d4a-ae7e-ee12f9eaef97 admin admin] PATH_INFO: `/v3/projects/892fe3caf9634fb78a40188f558be562/users/e6aa19872bd3486da5c81caeec578861/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:59.958066 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12cc1ab-2e89-4d4a-ae7e-ee12f9eaef97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=892fe3caf9634fb78a40188f558be562, user_id=e6aa19872bd3486da5c81caeec578861, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:59.960881 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12cc1ab-2e89-4d4a-ae7e-ee12f9eaef97 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:13:59.972059 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 932/1855] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:13:59 2022] PUT /identity/v3/projects/892fe3caf9634fb78a40188f558be562/users/e6aa19872bd3486da5c81caeec578861/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:13:59.974300 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6466d26-9126-4ad1-9829-165c83f76bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:13:59.974663 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12cc1ab-2e89-4d4a-ae7e-ee12f9eaef97 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:13:59.975068 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6466d26-9126-4ad1-9829-165c83f76bdc admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:13:59.975408 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6466d26-9126-4ad1-9829-165c83f76bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:13:59.975741 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6466d26-9126-4ad1-9829-165c83f76bdc admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:13:59.976589 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6466d26-9126-4ad1-9829-165c83f76bdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:13:59.978660 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6466d26-9126-4ad1-9829-165c83f76bdc admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:00.002615 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 924/1856] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:13:59 2022] POST /identity/v3/users => generated 299 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:14:00.005032 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe44038-47c2-4804-bde5-2ff58784569c None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:00.005538 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe44038-47c2-4804-bde5-2ff58784569c None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:00.006048 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe44038-47c2-4804-bde5-2ff58784569c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:00.036588 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-ffe44038-47c2-4804-bde5-2ff58784569c None None] MFA Rules not processed for user `e6aa19872bd3486da5c81caeec578861`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:00.037803 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-ffe44038-47c2-4804-bde5-2ff58784569c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:00.042840 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12cc1ab-2e89-4d4a-ae7e-ee12f9eaef97 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:00.049241 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f8a600-bf4f-4547-bc87-2142844e885f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:00.049856 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f8a600-bf4f-4547-bc87-2142844e885f admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:00.050252 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f8a600-bf4f-4547-bc87-2142844e885f admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:00.050528 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f8a600-bf4f-4547-bc87-2142844e885f admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:00.051204 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f8a600-bf4f-4547-bc87-2142844e885f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:00.052979 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f8a600-bf4f-4547-bc87-2142844e885f admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:00.066678 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 933/1857] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:14:00.066678 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f8a600-bf4f-4547-bc87-2142844e885f admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:00.073780 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:00.080526 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f8a600-bf4f-4547-bc87-2142844e885f admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:00.087657 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb7ad34-6ab4-4740-b475-f6feedef1aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:00.088303 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb7ad34-6ab4-4740-b475-f6feedef1aca admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:00.088574 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb7ad34-6ab4-4740-b475-f6feedef1aca admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:00.088848 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb7ad34-6ab4-4740-b475-f6feedef1aca admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:00.089610 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb7ad34-6ab4-4740-b475-f6feedef1aca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:00.091091 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb7ad34-6ab4-4740-b475-f6feedef1aca admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:00.103810 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 934/1858] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:14:00.104651 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb7ad34-6ab4-4740-b475-f6feedef1aca admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:00.115353 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb7ad34-6ab4-4740-b475-f6feedef1aca admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:00.120234 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666155ca-f8eb-489c-a39f-e7a67205ff3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:00.120835 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666155ca-f8eb-489c-a39f-e7a67205ff3a admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:00.121112 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666155ca-f8eb-489c-a39f-e7a67205ff3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:00.121806 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 925/1859] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:00 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:00.122670 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe44038-47c2-4804-bde5-2ff58784569c None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:00.124027 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666155ca-f8eb-489c-a39f-e7a67205ff3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:00.124783 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666155ca-f8eb-489c-a39f-e7a67205ff3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:00.126787 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666155ca-f8eb-489c-a39f-e7a67205ff3a admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:00.138908 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe44038-47c2-4804-bde5-2ff58784569c None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:00.144252 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 935/1860] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:14:00.144252 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666155ca-f8eb-489c-a39f-e7a67205ff3a admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:00.145685 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-666155ca-f8eb-489c-a39f-e7a67205ff3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:00.148303 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc66693-4551-49fd-9af6-f234038bbbab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:00.149642 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc66693-4551-49fd-9af6-f234038bbbab admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:00.149642 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc66693-4551-49fd-9af6-f234038bbbab admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:00.150030 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc66693-4551-49fd-9af6-f234038bbbab admin admin] PATH_INFO: `/v3/projects/91be089a98664017a69d71446142a61c/users/3d9b1dd73605493da6a6f93f3bff4fd9/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:00.155646 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:00.167837 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc66693-4551-49fd-9af6-f234038bbbab admin admin] RBAC: Authorizing `identity:create_grant(project_id=91be089a98664017a69d71446142a61c, user_id=3d9b1dd73605493da6a6f93f3bff4fd9, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:00.170707 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc66693-4551-49fd-9af6-f234038bbbab admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:00.188832 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 926/1861] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:14:00 2022] PUT /identity/v3/projects/91be089a98664017a69d71446142a61c/users/3d9b1dd73605493da6a6f93f3bff4fd9/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:00.189875 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc66693-4551-49fd-9af6-f234038bbbab admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:00.217032 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666155ca-f8eb-489c-a39f-e7a67205ff3a admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:00.270608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc66693-4551-49fd-9af6-f234038bbbab admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:00.276590 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b315e-b068-43db-8a98-546ccb2b326a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:00.277420 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b315e-b068-43db-8a98-546ccb2b326a service cinder] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:00.277537 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01646eb7-7b54-404e-81de-8c37ef82a69d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:00.277977 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b315e-b068-43db-8a98-546ccb2b326a service cinder] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:00.278153 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b315e-b068-43db-8a98-546ccb2b326a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:00.278437 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01646eb7-7b54-404e-81de-8c37ef82a69d admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:00.278793 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01646eb7-7b54-404e-81de-8c37ef82a69d admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:00.279150 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01646eb7-7b54-404e-81de-8c37ef82a69d admin admin] PATH_INFO: `/v3/projects/031b416f3b65423d98f2c4f25c7fd9fa/users/1fccb650c0f140109ea5be90a7651475/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:00.297709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01646eb7-7b54-404e-81de-8c37ef82a69d admin admin] RBAC: Authorizing `identity:create_grant(project_id=031b416f3b65423d98f2c4f25c7fd9fa, user_id=1fccb650c0f140109ea5be90a7651475, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:00.300822 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01646eb7-7b54-404e-81de-8c37ef82a69d admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:00.315934 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 927/1862] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:14:00 2022] PUT /identity/v3/projects/031b416f3b65423d98f2c4f25c7fd9fa/users/1fccb650c0f140109ea5be90a7651475/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:00.316891 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01646eb7-7b54-404e-81de-8c37ef82a69d admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:00.352219 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3b315e-b068-43db-8a98-546ccb2b326a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:00.353654 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3b315e-b068-43db-8a98-546ccb2b326a service cinder] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:00.369638 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 936/1863] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:14:00 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:14:00.370945 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69339a6c-0c2a-4741-a19c-0d5ab7da7a17 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:00.371232 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69339a6c-0c2a-4741-a19c-0d5ab7da7a17 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:00.371508 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69339a6c-0c2a-4741-a19c-0d5ab7da7a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:00.406974 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01646eb7-7b54-404e-81de-8c37ef82a69d admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:00.412885 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31cac99-2034-495a-8588-2817d79b4e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:00.413758 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31cac99-2034-495a-8588-2817d79b4e8f admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:00.414106 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31cac99-2034-495a-8588-2817d79b4e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:00.414443 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31cac99-2034-495a-8588-2817d79b4e8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:00.415159 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31cac99-2034-495a-8588-2817d79b4e8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:00.417163 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31cac99-2034-495a-8588-2817d79b4e8f admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:00.425328 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-69339a6c-0c2a-4741-a19c-0d5ab7da7a17 None None] MFA Rules not processed for user `1fccb650c0f140109ea5be90a7651475`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:00.426638 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-69339a6c-0c2a-4741-a19c-0d5ab7da7a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:00.429996 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 928/1864] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:14:00.433844 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31cac99-2034-495a-8588-2817d79b4e8f admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:00.466067 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31cac99-2034-495a-8588-2817d79b4e8f admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:00.485731 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4e2cee-df17-4d73-a208-e1840929e148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:00.486446 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4e2cee-df17-4d73-a208-e1840929e148 admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:00.486741 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4e2cee-df17-4d73-a208-e1840929e148 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:00.487005 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4e2cee-df17-4d73-a208-e1840929e148 admin admin] PATH_INFO: `/v3/projects/91be089a98664017a69d71446142a61c/users/3d9b1dd73605493da6a6f93f3bff4fd9/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:00.500371 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4e2cee-df17-4d73-a208-e1840929e148 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91be089a98664017a69d71446142a61c, user_id=3d9b1dd73605493da6a6f93f3bff4fd9, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:00.520486 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4e2cee-df17-4d73-a208-e1840929e148 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:00.520593 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 937/1865] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:00.529029 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 929/1866] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:14:00 2022] PUT /identity/v3/projects/91be089a98664017a69d71446142a61c/users/3d9b1dd73605493da6a6f93f3bff4fd9/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:14:00.533092 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69339a6c-0c2a-4741-a19c-0d5ab7da7a17 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:00.537996 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4e2cee-df17-4d73-a208-e1840929e148 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:00.540685 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:00.545657 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69339a6c-0c2a-4741-a19c-0d5ab7da7a17 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:00.554932 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:00.606558 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4e2cee-df17-4d73-a208-e1840929e148 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:00.610663 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7160cac8-2680-497b-80f6-714e27dc3762 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:00.611269 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7160cac8-2680-497b-80f6-714e27dc3762 service cinder] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:00.611547 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7160cac8-2680-497b-80f6-714e27dc3762 service cinder] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:00.611820 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7160cac8-2680-497b-80f6-714e27dc3762 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:00.613671 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-7160cac8-2680-497b-80f6-714e27dc3762 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:00.614717 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944c09dc-392c-4382-89fd-ef7ab2d1d8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:00.615327 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c09dc-392c-4382-89fd-ef7ab2d1d8d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:00.615607 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c09dc-392c-4382-89fd-ef7ab2d1d8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:00.615880 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c09dc-392c-4382-89fd-ef7ab2d1d8d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:00.616594 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944c09dc-392c-4382-89fd-ef7ab2d1d8d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:00.618570 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944c09dc-392c-4382-89fd-ef7ab2d1d8d9 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:00.651986 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 938/1867] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:14:00 2022] POST /identity/v3/projects => generated 369 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:14:00.651986 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e9ae9-20f6-4057-a624-c0c074c608a6 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:00.651986 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e9ae9-20f6-4057-a624-c0c074c608a6 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:00.651986 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e9ae9-20f6-4057-a624-c0c074c608a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:00.678571 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-487e9ae9-20f6-4057-a624-c0c074c608a6 None None] MFA Rules not processed for user `3d9b1dd73605493da6a6f93f3bff4fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:00.679771 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-487e9ae9-20f6-4057-a624-c0c074c608a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:00.684631 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7160cac8-2680-497b-80f6-714e27dc3762 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:00.686073 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7160cac8-2680-497b-80f6-714e27dc3762 service cinder] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:00.696397 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 930/1868] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:14:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:14:00.697140 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7160cac8-2680-497b-80f6-714e27dc3762 service cinder] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:00.708086 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7160cac8-2680-497b-80f6-714e27dc3762 service cinder] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:00.758966 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 939/1869] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:00 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:00.768188 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487e9ae9-20f6-4057-a624-c0c074c608a6 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:00.769688 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-487e9ae9-20f6-4057-a624-c0c074c608a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:00.772090 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33314be-d015-4620-8138-c47c56d5d04e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:00.773009 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33314be-d015-4620-8138-c47c56d5d04e admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:00.773009 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33314be-d015-4620-8138-c47c56d5d04e admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:00.773286 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33314be-d015-4620-8138-c47c56d5d04e admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:00.773902 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33314be-d015-4620-8138-c47c56d5d04e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:00.775826 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33314be-d015-4620-8138-c47c56d5d04e admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:00.804020 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 931/1870] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:14:00 2022] POST /identity/v3/users => generated 291 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:14:00.805472 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33314be-d015-4620-8138-c47c56d5d04e admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:00.820742 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33314be-d015-4620-8138-c47c56d5d04e admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:00.827420 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1a778a-bd63-46a9-976c-fca897b3b9b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:00.829283 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a778a-bd63-46a9-976c-fca897b3b9b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:00.829595 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a778a-bd63-46a9-976c-fca897b3b9b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:00.829952 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a778a-bd63-46a9-976c-fca897b3b9b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:00.830993 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1a778a-bd63-46a9-976c-fca897b3b9b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:00.833512 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1a778a-bd63-46a9-976c-fca897b3b9b9 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:00.850188 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487e9ae9-20f6-4057-a624-c0c074c608a6 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:00.852462 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 932/1871] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:14:00.854668 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e07270-fae6-4b2f-a96c-91303bdc1b95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:00.855419 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e07270-fae6-4b2f-a96c-91303bdc1b95 service cinder] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:00.855818 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e07270-fae6-4b2f-a96c-91303bdc1b95 service cinder] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:00.855927 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1a778a-bd63-46a9-976c-fca897b3b9b9 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:00.856285 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e07270-fae6-4b2f-a96c-91303bdc1b95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:00.871582 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1a778a-bd63-46a9-976c-fca897b3b9b9 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:00.877746 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70f95a4-82e6-4bca-9e34-5bbba4894465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:00.878552 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f95a4-82e6-4bca-9e34-5bbba4894465 admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:00.879055 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f95a4-82e6-4bca-9e34-5bbba4894465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:00.879467 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f95a4-82e6-4bca-9e34-5bbba4894465 admin admin] PATH_INFO: `/v3/projects/acf3a9a0cfc8460c9ef2bdc1319f60a6/users/c474cb9fc2bb4ca18a6c85df91f4c029/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:00.899353 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:00.906639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70f95a4-82e6-4bca-9e34-5bbba4894465 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acf3a9a0cfc8460c9ef2bdc1319f60a6, user_id=c474cb9fc2bb4ca18a6c85df91f4c029, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:00.911099 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70f95a4-82e6-4bca-9e34-5bbba4894465 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:00.936238 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 933/1872] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:14:00 2022] PUT /identity/v3/projects/acf3a9a0cfc8460c9ef2bdc1319f60a6/users/c474cb9fc2bb4ca18a6c85df91f4c029/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:00.939155 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70f95a4-82e6-4bca-9e34-5bbba4894465 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:00.951380 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e07270-fae6-4b2f-a96c-91303bdc1b95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:00.953738 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e07270-fae6-4b2f-a96c-91303bdc1b95 service cinder] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:00.975620 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 940/1873] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:14:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 01:14:00.986538 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:01.062687 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70f95a4-82e6-4bca-9e34-5bbba4894465 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:01.068398 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77924ba-2517-47e4-b686-532c4ac3170f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:01.070568 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77924ba-2517-47e4-b686-532c4ac3170f admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:01.070874 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77924ba-2517-47e4-b686-532c4ac3170f admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:01.071244 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77924ba-2517-47e4-b686-532c4ac3170f admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:01.071885 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77924ba-2517-47e4-b686-532c4ac3170f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:01.074213 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77924ba-2517-47e4-b686-532c4ac3170f admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:01.092162 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 934/1874] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:14:01.092670 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e07270-fae6-4b2f-a96c-91303bdc1b95 service cinder] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:01.120073 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e07270-fae6-4b2f-a96c-91303bdc1b95 service cinder] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:01.126322 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2255a3c7-1d5a-40f5-a473-19ad4a6df1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:01.127012 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2255a3c7-1d5a-40f5-a473-19ad4a6df1cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:01.127294 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2255a3c7-1d5a-40f5-a473-19ad4a6df1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:01.127643 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2255a3c7-1d5a-40f5-a473-19ad4a6df1cf admin admin] PATH_INFO: `/v3/projects/acf3a9a0cfc8460c9ef2bdc1319f60a6/users/c474cb9fc2bb4ca18a6c85df91f4c029/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:01.130116 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2255a3c7-1d5a-40f5-a473-19ad4a6df1cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=acf3a9a0cfc8460c9ef2bdc1319f60a6, user_id=c474cb9fc2bb4ca18a6c85df91f4c029, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:01.133709 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2255a3c7-1d5a-40f5-a473-19ad4a6df1cf admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:01.147085 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 941/1875] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:14:01 2022] PUT /identity/v3/projects/acf3a9a0cfc8460c9ef2bdc1319f60a6/users/c474cb9fc2bb4ca18a6c85df91f4c029/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:14:01.158472 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5ca88-a438-46b1-af1c-51e47007408c None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:01.158859 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5ca88-a438-46b1-af1c-51e47007408c None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:01.159141 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5ca88-a438-46b1-af1c-51e47007408c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:01.189812 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-c9a5ca88-a438-46b1-af1c-51e47007408c None None] MFA Rules not processed for user `c474cb9fc2bb4ca18a6c85df91f4c029`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:01.191321 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-c9a5ca88-a438-46b1-af1c-51e47007408c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:01.269837 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 935/1876] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:01 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:01.281450 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2255a3c7-1d5a-40f5-a473-19ad4a6df1cf admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:01.356091 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2255a3c7-1d5a-40f5-a473-19ad4a6df1cf admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:01.361135 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6af9d6-b84e-49d8-97bc-30c3788415a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:01.361831 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6af9d6-b84e-49d8-97bc-30c3788415a4 service cinder] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:01.362181 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6af9d6-b84e-49d8-97bc-30c3788415a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:01.362677 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6af9d6-b84e-49d8-97bc-30c3788415a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:01.383986 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6af9d6-b84e-49d8-97bc-30c3788415a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:01.385800 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6af9d6-b84e-49d8-97bc-30c3788415a4 service cinder] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:01.415731 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 942/1877] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:14:01 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:14:02.857152 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a5ca88-a438-46b1-af1c-51e47007408c None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:02.921893 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a5ca88-a438-46b1-af1c-51e47007408c None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:02.926566 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec13b71b-e49c-43e9-a20b-8a5983dc86de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:02.927191 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13b71b-e49c-43e9-a20b-8a5983dc86de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:02.927472 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13b71b-e49c-43e9-a20b-8a5983dc86de admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:02.927770 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13b71b-e49c-43e9-a20b-8a5983dc86de admin admin] PATH_INFO: `/v3/users/1fccb650c0f140109ea5be90a7651475` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:02.929165 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec13b71b-e49c-43e9-a20b-8a5983dc86de admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fccb650c0f140109ea5be90a7651475)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:02.931241 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec13b71b-e49c-43e9-a20b-8a5983dc86de admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:02.978333 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:02.980331 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-ec13b71b-e49c-43e9-a20b-8a5983dc86de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fccb650c0f140109ea5be90a7651475'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:02.993553 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-ec13b71b-e49c-43e9-a20b-8a5983dc86de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fccb650c0f140109ea5be90a7651475'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:03.001164 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-ec13b71b-e49c-43e9-a20b-8a5983dc86de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fccb650c0f140109ea5be90a7651475'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:03.001870 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-ec13b71b-e49c-43e9-a20b-8a5983dc86de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fccb650c0f140109ea5be90a7651475'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:03.009968 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-ec13b71b-e49c-43e9-a20b-8a5983dc86de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fccb650c0f140109ea5be90a7651475'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:03.017734 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 936/1878] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:14:02 2022] DELETE /identity/v3/users/1fccb650c0f140109ea5be90a7651475 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:03.022603 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09976983-a73a-4525-8ba7-cc54454a4a62 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:03.023108 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09976983-a73a-4525-8ba7-cc54454a4a62 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:03.023239 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09976983-a73a-4525-8ba7-cc54454a4a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:03.042565 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-09976983-a73a-4525-8ba7-cc54454a4a62 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:03.043737 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-09976983-a73a-4525-8ba7-cc54454a4a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:03.103972 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 943/1879] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:03.115396 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec13b71b-e49c-43e9-a20b-8a5983dc86de admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:03.116958 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-ec13b71b-e49c-43e9-a20b-8a5983dc86de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:03.179313 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec13b71b-e49c-43e9-a20b-8a5983dc86de admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:03.182957 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5baf7d-9b97-4a5d-916c-c29fac569109 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:03.183559 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5baf7d-9b97-4a5d-916c-c29fac569109 service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:03.183912 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5baf7d-9b97-4a5d-916c-c29fac569109 service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:03.184097 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5baf7d-9b97-4a5d-916c-c29fac569109 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:03.193039 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5baf7d-9b97-4a5d-916c-c29fac569109 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:03.194481 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5baf7d-9b97-4a5d-916c-c29fac569109 service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:03.205060 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 937/1880] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:14:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:14:03.572286 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09976983-a73a-4525-8ba7-cc54454a4a62 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:03.573710 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-09976983-a73a-4525-8ba7-cc54454a4a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:03.591833 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09976983-a73a-4525-8ba7-cc54454a4a62 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:03.597713 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54ad2d4-2ac0-4d79-b84d-47a393a60c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:03.598348 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ad2d4-2ac0-4d79-b84d-47a393a60c23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:03.598739 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ad2d4-2ac0-4d79-b84d-47a393a60c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:03.599014 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ad2d4-2ac0-4d79-b84d-47a393a60c23 admin admin] PATH_INFO: `/v3/projects/031b416f3b65423d98f2c4f25c7fd9fa` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:03.600113 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54ad2d4-2ac0-4d79-b84d-47a393a60c23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=031b416f3b65423d98f2c4f25c7fd9fa)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:03.602120 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54ad2d4-2ac0-4d79-b84d-47a393a60c23 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:03.653715 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-e54ad2d4-2ac0-4d79-b84d-47a393a60c23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '031b416f3b65423d98f2c4f25c7fd9fa', 'request_id': 'req-e54ad2d4-2ac0-4d79-b84d-47a393a60c23'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:03.656503 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:03.662311 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-e54ad2d4-2ac0-4d79-b84d-47a393a60c23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '031b416f3b65423d98f2c4f25c7fd9fa', 'request_id': 'req-e54ad2d4-2ac0-4d79-b84d-47a393a60c23'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:03.669115 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-e54ad2d4-2ac0-4d79-b84d-47a393a60c23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '031b416f3b65423d98f2c4f25c7fd9fa', 'request_id': 'req-e54ad2d4-2ac0-4d79-b84d-47a393a60c23'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:03.682520 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-e54ad2d4-2ac0-4d79-b84d-47a393a60c23 admin admin] The token cache is being invalidate because project 031b416f3b65423d98f2c4f25c7fd9fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:14:03.682816 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-e54ad2d4-2ac0-4d79-b84d-47a393a60c23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:03.683443 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-e54ad2d4-2ac0-4d79-b84d-47a393a60c23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:03.684999 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 944/1881] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:14:03 2022] DELETE /identity/v3/projects/031b416f3b65423d98f2c4f25c7fd9fa => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:03.689124 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5baf7d-9b97-4a5d-916c-c29fac569109 service neutron] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:03.699585 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5baf7d-9b97-4a5d-916c-c29fac569109 service neutron] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:03.758801 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be31398b-9895-4469-a221-514572b4e401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:03.759697 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be31398b-9895-4469-a221-514572b4e401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:03.759979 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be31398b-9895-4469-a221-514572b4e401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:03.760291 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be31398b-9895-4469-a221-514572b4e401 admin admin] PATH_INFO: `/v3/users/c474cb9fc2bb4ca18a6c85df91f4c029` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:03.761420 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be31398b-9895-4469-a221-514572b4e401 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c474cb9fc2bb4ca18a6c85df91f4c029)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:03.763393 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be31398b-9895-4469-a221-514572b4e401 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:03.816100 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-be31398b-9895-4469-a221-514572b4e401 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c474cb9fc2bb4ca18a6c85df91f4c029'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:03.828949 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-be31398b-9895-4469-a221-514572b4e401 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c474cb9fc2bb4ca18a6c85df91f4c029'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:03.830948 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:03.836695 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-be31398b-9895-4469-a221-514572b4e401 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c474cb9fc2bb4ca18a6c85df91f4c029'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:03.838120 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-be31398b-9895-4469-a221-514572b4e401 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c474cb9fc2bb4ca18a6c85df91f4c029'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:03.846070 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-be31398b-9895-4469-a221-514572b4e401 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c474cb9fc2bb4ca18a6c85df91f4c029'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:03.852274 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 938/1882] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:14:03 2022] DELETE /identity/v3/users/c474cb9fc2bb4ca18a6c85df91f4c029 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:04.193884 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54ad2d4-2ac0-4d79-b84d-47a393a60c23 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:04.195539 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-e54ad2d4-2ac0-4d79-b84d-47a393a60c23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:04.209826 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54ad2d4-2ac0-4d79-b84d-47a393a60c23 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:04.285302 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bde34fc-4c06-4b9b-9398-6c2168b2772f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:04.286250 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde34fc-4c06-4b9b-9398-6c2168b2772f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:04.286536 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde34fc-4c06-4b9b-9398-6c2168b2772f admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:04.286812 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde34fc-4c06-4b9b-9398-6c2168b2772f admin admin] PATH_INFO: `/v3/projects/acf3a9a0cfc8460c9ef2bdc1319f60a6` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:04.288006 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bde34fc-4c06-4b9b-9398-6c2168b2772f admin admin] RBAC: Authorizing `identity:delete_project(project_id=acf3a9a0cfc8460c9ef2bdc1319f60a6)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:04.290004 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bde34fc-4c06-4b9b-9398-6c2168b2772f admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:04.346686 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-0bde34fc-4c06-4b9b-9398-6c2168b2772f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acf3a9a0cfc8460c9ef2bdc1319f60a6', 'request_id': 'req-0bde34fc-4c06-4b9b-9398-6c2168b2772f'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:04.353439 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-0bde34fc-4c06-4b9b-9398-6c2168b2772f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acf3a9a0cfc8460c9ef2bdc1319f60a6', 'request_id': 'req-0bde34fc-4c06-4b9b-9398-6c2168b2772f'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:04.359823 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-0bde34fc-4c06-4b9b-9398-6c2168b2772f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acf3a9a0cfc8460c9ef2bdc1319f60a6', 'request_id': 'req-0bde34fc-4c06-4b9b-9398-6c2168b2772f'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:04.372914 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-0bde34fc-4c06-4b9b-9398-6c2168b2772f admin admin] The token cache is being invalidate because project acf3a9a0cfc8460c9ef2bdc1319f60a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:14:04.373206 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-0bde34fc-4c06-4b9b-9398-6c2168b2772f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:04.373842 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-0bde34fc-4c06-4b9b-9398-6c2168b2772f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:04.376175 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 945/1883] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:14:04 2022] DELETE /identity/v3/projects/acf3a9a0cfc8460c9ef2bdc1319f60a6 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:04.381148 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482fe24c-9234-4c4d-be0c-e1c0820a0e02 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:04.381629 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482fe24c-9234-4c4d-be0c-e1c0820a0e02 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:04.381754 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482fe24c-9234-4c4d-be0c-e1c0820a0e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:04.402376 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-482fe24c-9234-4c4d-be0c-e1c0820a0e02 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:04.403601 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-482fe24c-9234-4c4d-be0c-e1c0820a0e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:04.464339 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 939/1884] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:04.472766 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba2fef-f3c9-4e0d-8855-f862909e667b None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:04.473054 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba2fef-f3c9-4e0d-8855-f862909e667b None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:04.473335 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba2fef-f3c9-4e0d-8855-f862909e667b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:04.495921 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-c9ba2fef-f3c9-4e0d-8855-f862909e667b None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:04.497273 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-c9ba2fef-f3c9-4e0d-8855-f862909e667b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:04.508418 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 946/1885] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:04.510922 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482fe24c-9234-4c4d-be0c-e1c0820a0e02 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:04.522425 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482fe24c-9234-4c4d-be0c-e1c0820a0e02 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:04.528432 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfad3aa-e76f-4318-8209-2cd671c6c9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:04.529167 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfad3aa-e76f-4318-8209-2cd671c6c9aa admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:04.529454 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfad3aa-e76f-4318-8209-2cd671c6c9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:04.529731 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfad3aa-e76f-4318-8209-2cd671c6c9aa admin admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:04.530392 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfad3aa-e76f-4318-8209-2cd671c6c9aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:14:04.530698 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfad3aa-e76f-4318-8209-2cd671c6c9aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:04.532436 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfad3aa-e76f-4318-8209-2cd671c6c9aa admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:04.555644 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 940/1886] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:14:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:14:04.560005 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ba2fef-f3c9-4e0d-8855-f862909e667b None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:04.570864 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ba2fef-f3c9-4e0d-8855-f862909e667b None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:04.576377 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b1485c-d537-4cfa-aeb3-494a3059f705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:04.577089 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1485c-d537-4cfa-aeb3-494a3059f705 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:04.577466 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1485c-d537-4cfa-aeb3-494a3059f705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:04.578468 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1485c-d537-4cfa-aeb3-494a3059f705 admin admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:04.579407 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b1485c-d537-4cfa-aeb3-494a3059f705 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:04.581443 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b1485c-d537-4cfa-aeb3-494a3059f705 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:04.613995 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 947/1887] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:14:04 2022] POST /identity/v3/projects => generated 381 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:14:04.615137 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfad3aa-e76f-4318-8209-2cd671c6c9aa admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:04.625458 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfad3aa-e76f-4318-8209-2cd671c6c9aa admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:04.685519 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c225124-6809-48c4-8e92-2c0d72c1baf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:04.686125 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c225124-6809-48c4-8e92-2c0d72c1baf0 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:04.686409 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c225124-6809-48c4-8e92-2c0d72c1baf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:04.686699 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c225124-6809-48c4-8e92-2c0d72c1baf0 admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:04.687569 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c225124-6809-48c4-8e92-2c0d72c1baf0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:04.689547 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c225124-6809-48c4-8e92-2c0d72c1baf0 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:04.710925 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 941/1888] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:14:04 2022] POST /identity/v3/users => generated 291 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:14:04.714991 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b1485c-d537-4cfa-aeb3-494a3059f705 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:04.726308 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b1485c-d537-4cfa-aeb3-494a3059f705 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:04.734657 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bc68f7-1312-4cc1-b283-ad02fc3bcf22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:04.735274 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bc68f7-1312-4cc1-b283-ad02fc3bcf22 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:04.735557 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bc68f7-1312-4cc1-b283-ad02fc3bcf22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:04.735829 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bc68f7-1312-4cc1-b283-ad02fc3bcf22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:04.736508 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bc68f7-1312-4cc1-b283-ad02fc3bcf22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:04.738120 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bc68f7-1312-4cc1-b283-ad02fc3bcf22 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:04.748447 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 948/1889] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:14:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:14:04.752614 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c225124-6809-48c4-8e92-2c0d72c1baf0 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:04.762805 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c225124-6809-48c4-8e92-2c0d72c1baf0 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:04.768393 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b646ec69-5d41-46dd-8d17-5853981a4c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:04.768988 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b646ec69-5d41-46dd-8d17-5853981a4c63 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:04.769299 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b646ec69-5d41-46dd-8d17-5853981a4c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:04.769607 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b646ec69-5d41-46dd-8d17-5853981a4c63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:04.770318 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b646ec69-5d41-46dd-8d17-5853981a4c63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:04.771910 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b646ec69-5d41-46dd-8d17-5853981a4c63 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:04.782993 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 942/1890] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:14:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:14:04.787035 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bc68f7-1312-4cc1-b283-ad02fc3bcf22 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:04.797055 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bc68f7-1312-4cc1-b283-ad02fc3bcf22 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:04.801642 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3922b71-7d71-4331-84dc-6ee730a626f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:04.802289 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3922b71-7d71-4331-84dc-6ee730a626f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:04.802573 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3922b71-7d71-4331-84dc-6ee730a626f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:04.802851 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3922b71-7d71-4331-84dc-6ee730a626f3 admin admin] PATH_INFO: `/v3/projects/0ed48d92010942a882652234d1cb7f59/users/a087a59950374197a928b735718d78b5/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:04.817430 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3922b71-7d71-4331-84dc-6ee730a626f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ed48d92010942a882652234d1cb7f59, user_id=a087a59950374197a928b735718d78b5, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:04.820155 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3922b71-7d71-4331-84dc-6ee730a626f3 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:04.834195 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 949/1891] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:14:04 2022] PUT /identity/v3/projects/0ed48d92010942a882652234d1cb7f59/users/a087a59950374197a928b735718d78b5/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:04.845812 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e5382a-f719-4633-9004-e8414896c0d9 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:04.846097 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e5382a-f719-4633-9004-e8414896c0d9 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:04.846371 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e5382a-f719-4633-9004-e8414896c0d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:04.875319 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-29e5382a-f719-4633-9004-e8414896c0d9 None None] MFA Rules not processed for user `a087a59950374197a928b735718d78b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:04.876483 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-29e5382a-f719-4633-9004-e8414896c0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:04.918665 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:04.946664 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 943/1892] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:04 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:04.951120 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3922b71-7d71-4331-84dc-6ee730a626f3 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:04.960476 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:05.019257 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3922b71-7d71-4331-84dc-6ee730a626f3 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:05.024187 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6883a703-4231-487c-bf24-913b296bdc7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:05.024794 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883a703-4231-487c-bf24-913b296bdc7e admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:05.025072 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883a703-4231-487c-bf24-913b296bdc7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:05.025374 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883a703-4231-487c-bf24-913b296bdc7e admin admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:05.025996 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6883a703-4231-487c-bf24-913b296bdc7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:05.027936 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6883a703-4231-487c-bf24-913b296bdc7e admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:05.054321 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 950/1893] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:14:04 2022] POST /identity/v3/projects => generated 381 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:14:05.058366 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e5382a-f719-4633-9004-e8414896c0d9 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:05.070037 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e5382a-f719-4633-9004-e8414896c0d9 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:05.127673 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e75f34-752f-44b7-9bae-05c8bfcd8ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:05.128231 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e75f34-752f-44b7-9bae-05c8bfcd8ea8 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:05.128510 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e75f34-752f-44b7-9bae-05c8bfcd8ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:05.128779 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e75f34-752f-44b7-9bae-05c8bfcd8ea8 admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:05.129540 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e75f34-752f-44b7-9bae-05c8bfcd8ea8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:05.131501 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e75f34-752f-44b7-9bae-05c8bfcd8ea8 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:05.159516 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 944/1894] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:14:05 2022] POST /identity/v3/users => generated 297 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:14:05.161227 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6883a703-4231-487c-bf24-913b296bdc7e admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:05.190957 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6883a703-4231-487c-bf24-913b296bdc7e admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:05.197337 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a508438c-d171-42fb-aef5-b39be537a316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:05.198660 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a508438c-d171-42fb-aef5-b39be537a316 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:05.199858 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a508438c-d171-42fb-aef5-b39be537a316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:05.200244 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a508438c-d171-42fb-aef5-b39be537a316 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:05.200993 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a508438c-d171-42fb-aef5-b39be537a316 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:05.202715 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a508438c-d171-42fb-aef5-b39be537a316 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:05.218361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 951/1895] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:14:05 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:14:05.220358 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e75f34-752f-44b7-9bae-05c8bfcd8ea8 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:05.231786 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e75f34-752f-44b7-9bae-05c8bfcd8ea8 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:05.237456 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344865b4-cc28-408f-a004-764af57eaf78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:05.238375 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344865b4-cc28-408f-a004-764af57eaf78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:05.238722 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344865b4-cc28-408f-a004-764af57eaf78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:05.239051 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344865b4-cc28-408f-a004-764af57eaf78 admin admin] PATH_INFO: `/v3/projects/d008c6a4bc8b4b8d98719bb402872a31/users/1b823e57eab54a9ea1e0f21e94931a73/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:05.257561 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344865b4-cc28-408f-a004-764af57eaf78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d008c6a4bc8b4b8d98719bb402872a31, user_id=1b823e57eab54a9ea1e0f21e94931a73, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:05.260658 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344865b4-cc28-408f-a004-764af57eaf78 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:05.279642 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 945/1896] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:14:05 2022] PUT /identity/v3/projects/d008c6a4bc8b4b8d98719bb402872a31/users/1b823e57eab54a9ea1e0f21e94931a73/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:05.282081 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a508438c-d171-42fb-aef5-b39be537a316 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:05.292646 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a508438c-d171-42fb-aef5-b39be537a316 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:05.363103 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728f987e-9543-4621-a5f1-d7829fdd9806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:05.363872 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f987e-9543-4621-a5f1-d7829fdd9806 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:05.364317 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f987e-9543-4621-a5f1-d7829fdd9806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:05.364657 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f987e-9543-4621-a5f1-d7829fdd9806 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:05.365416 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728f987e-9543-4621-a5f1-d7829fdd9806 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:05.367316 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728f987e-9543-4621-a5f1-d7829fdd9806 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:05.380188 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 952/1897] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:14:05 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:14:05.384619 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344865b4-cc28-408f-a004-764af57eaf78 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:05.396441 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344865b4-cc28-408f-a004-764af57eaf78 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:05.401700 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce14c048-1a64-4b06-8f3d-1e7cdf197a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:05.402374 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce14c048-1a64-4b06-8f3d-1e7cdf197a7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:05.402655 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce14c048-1a64-4b06-8f3d-1e7cdf197a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:05.402927 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce14c048-1a64-4b06-8f3d-1e7cdf197a7b admin admin] PATH_INFO: `/v3/projects/d008c6a4bc8b4b8d98719bb402872a31/users/1b823e57eab54a9ea1e0f21e94931a73/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:05.404888 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce14c048-1a64-4b06-8f3d-1e7cdf197a7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d008c6a4bc8b4b8d98719bb402872a31, user_id=1b823e57eab54a9ea1e0f21e94931a73, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:05.407668 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce14c048-1a64-4b06-8f3d-1e7cdf197a7b admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:05.424669 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 946/1898] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:14:05 2022] PUT /identity/v3/projects/d008c6a4bc8b4b8d98719bb402872a31/users/1b823e57eab54a9ea1e0f21e94931a73/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:05.430085 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a7b76-b5cd-446f-956b-e2692441f660 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:05.430395 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a7b76-b5cd-446f-956b-e2692441f660 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:05.430691 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a7b76-b5cd-446f-956b-e2692441f660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:05.462378 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-cb1a7b76-b5cd-446f-956b-e2692441f660 None None] MFA Rules not processed for user `1b823e57eab54a9ea1e0f21e94931a73`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:05.463906 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-cb1a7b76-b5cd-446f-956b-e2692441f660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:05.561686 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 953/1899] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:05 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:05.574784 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce14c048-1a64-4b06-8f3d-1e7cdf197a7b admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:05.576903 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-ce14c048-1a64-4b06-8f3d-1e7cdf197a7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:05.662500 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce14c048-1a64-4b06-8f3d-1e7cdf197a7b admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:05.666605 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b765337-0312-4c69-888c-246048ed3da5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:05.667356 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b765337-0312-4c69-888c-246048ed3da5 service cinder] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:05.667665 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b765337-0312-4c69-888c-246048ed3da5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:05.667944 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b765337-0312-4c69-888c-246048ed3da5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:05.678325 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b765337-0312-4c69-888c-246048ed3da5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:05.679709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b765337-0312-4c69-888c-246048ed3da5 service cinder] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:05.692054 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 947/1900] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:14:05 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:14:06.103694 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1a7b76-b5cd-446f-956b-e2692441f660 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:06.109103 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:06.118369 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1a7b76-b5cd-446f-956b-e2692441f660 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:06.124228 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d698ec48-d831-4e43-8de8-f895c3e14751 tempest-VolumeQuotaClassesTest-2004346750 tempest-VolumeQuotaClassesTest-2004346750-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b823e57eab54a9ea1e0f21e94931a73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd008c6a4bc8b4b8d98719bb402872a31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:06.125128 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698ec48-d831-4e43-8de8-f895c3e14751 tempest-VolumeQuotaClassesTest-2004346750 tempest-VolumeQuotaClassesTest-2004346750-project-admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:06.125490 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698ec48-d831-4e43-8de8-f895c3e14751 tempest-VolumeQuotaClassesTest-2004346750 tempest-VolumeQuotaClassesTest-2004346750-project-admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:06.125802 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698ec48-d831-4e43-8de8-f895c3e14751 tempest-VolumeQuotaClassesTest-2004346750 tempest-VolumeQuotaClassesTest-2004346750-project-admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:06.126607 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d698ec48-d831-4e43-8de8-f895c3e14751 tempest-VolumeQuotaClassesTest-2004346750 tempest-VolumeQuotaClassesTest-2004346750-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:14:06.127132 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d698ec48-d831-4e43-8de8-f895c3e14751 tempest-VolumeQuotaClassesTest-2004346750 tempest-VolumeQuotaClassesTest-2004346750-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:06.128917 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d698ec48-d831-4e43-8de8-f895c3e14751 tempest-VolumeQuotaClassesTest-2004346750 tempest-VolumeQuotaClassesTest-2004346750-project-admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:06.155905 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 954/1901] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:14:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:14:06.156866 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b765337-0312-4c69-888c-246048ed3da5 service cinder] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:06.173203 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b765337-0312-4c69-888c-246048ed3da5 service cinder] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:06.182244 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46cb392-b092-494f-93d2-509770a89074 tempest-VolumeQuotaClassesTest-2004346750 tempest-VolumeQuotaClassesTest-2004346750-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b823e57eab54a9ea1e0f21e94931a73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd008c6a4bc8b4b8d98719bb402872a31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:06.182979 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46cb392-b092-494f-93d2-509770a89074 tempest-VolumeQuotaClassesTest-2004346750 tempest-VolumeQuotaClassesTest-2004346750-project-admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:06.183348 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46cb392-b092-494f-93d2-509770a89074 tempest-VolumeQuotaClassesTest-2004346750 tempest-VolumeQuotaClassesTest-2004346750-project-admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:06.183646 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46cb392-b092-494f-93d2-509770a89074 tempest-VolumeQuotaClassesTest-2004346750 tempest-VolumeQuotaClassesTest-2004346750-project-admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:06.184477 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46cb392-b092-494f-93d2-509770a89074 tempest-VolumeQuotaClassesTest-2004346750 tempest-VolumeQuotaClassesTest-2004346750-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:06.186935 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46cb392-b092-494f-93d2-509770a89074 tempest-VolumeQuotaClassesTest-2004346750 tempest-VolumeQuotaClassesTest-2004346750-project-admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:06.221810 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 948/1902] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:14:06 2022] POST /identity/v3/projects => generated 354 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 01:14:06.226132 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d698ec48-d831-4e43-8de8-f895c3e14751 tempest-VolumeQuotaClassesTest-2004346750 tempest-VolumeQuotaClassesTest-2004346750-project-admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:06.248338 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d698ec48-d831-4e43-8de8-f895c3e14751 tempest-VolumeQuotaClassesTest-2004346750 tempest-VolumeQuotaClassesTest-2004346750-project-admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:06.314045 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec17b60-33e3-4400-9d59-f9da23be17df tempest-VolumeQuotaClassesTest-2004346750 tempest-VolumeQuotaClassesTest-2004346750-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b823e57eab54a9ea1e0f21e94931a73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd008c6a4bc8b4b8d98719bb402872a31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:06.314665 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec17b60-33e3-4400-9d59-f9da23be17df tempest-VolumeQuotaClassesTest-2004346750 tempest-VolumeQuotaClassesTest-2004346750-project-admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:06.314947 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec17b60-33e3-4400-9d59-f9da23be17df tempest-VolumeQuotaClassesTest-2004346750 tempest-VolumeQuotaClassesTest-2004346750-project-admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:06.315226 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec17b60-33e3-4400-9d59-f9da23be17df tempest-VolumeQuotaClassesTest-2004346750 tempest-VolumeQuotaClassesTest-2004346750-project-admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:06.315883 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec17b60-33e3-4400-9d59-f9da23be17df tempest-VolumeQuotaClassesTest-2004346750 tempest-VolumeQuotaClassesTest-2004346750-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:14:06.316199 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec17b60-33e3-4400-9d59-f9da23be17df tempest-VolumeQuotaClassesTest-2004346750 tempest-VolumeQuotaClassesTest-2004346750-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:06.317818 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec17b60-33e3-4400-9d59-f9da23be17df tempest-VolumeQuotaClassesTest-2004346750 tempest-VolumeQuotaClassesTest-2004346750-project-admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:06.331226 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 955/1903] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:14:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:14:06.375771 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46cb392-b092-494f-93d2-509770a89074 tempest-VolumeQuotaClassesTest-2004346750 tempest-VolumeQuotaClassesTest-2004346750-project-admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:06.388169 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46cb392-b092-494f-93d2-509770a89074 tempest-VolumeQuotaClassesTest-2004346750 tempest-VolumeQuotaClassesTest-2004346750-project-admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:06.395989 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22dd2d1-61f2-4acd-a3f8-441d63fdd52d tempest-VolumeQuotaClassesTest-2004346750 tempest-VolumeQuotaClassesTest-2004346750-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b823e57eab54a9ea1e0f21e94931a73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd008c6a4bc8b4b8d98719bb402872a31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:06.396667 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22dd2d1-61f2-4acd-a3f8-441d63fdd52d tempest-VolumeQuotaClassesTest-2004346750 tempest-VolumeQuotaClassesTest-2004346750-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:06.396953 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22dd2d1-61f2-4acd-a3f8-441d63fdd52d tempest-VolumeQuotaClassesTest-2004346750 tempest-VolumeQuotaClassesTest-2004346750-project-admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:06.397280 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22dd2d1-61f2-4acd-a3f8-441d63fdd52d tempest-VolumeQuotaClassesTest-2004346750 tempest-VolumeQuotaClassesTest-2004346750-project-admin] PATH_INFO: `/v3/projects/f2b04c8429ef46919ba62e863a0146d9` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:06.398572 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22dd2d1-61f2-4acd-a3f8-441d63fdd52d tempest-VolumeQuotaClassesTest-2004346750 tempest-VolumeQuotaClassesTest-2004346750-project-admin] RBAC: Authorizing `identity:delete_project(project_id=f2b04c8429ef46919ba62e863a0146d9)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:06.400536 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22dd2d1-61f2-4acd-a3f8-441d63fdd52d tempest-VolumeQuotaClassesTest-2004346750 tempest-VolumeQuotaClassesTest-2004346750-project-admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:06.414027 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:06.463220 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-a22dd2d1-61f2-4acd-a3f8-441d63fdd52d tempest-VolumeQuotaClassesTest-2004346750 tempest-VolumeQuotaClassesTest-2004346750-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2b04c8429ef46919ba62e863a0146d9', 'request_id': 'req-a22dd2d1-61f2-4acd-a3f8-441d63fdd52d'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:06.480942 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-a22dd2d1-61f2-4acd-a3f8-441d63fdd52d tempest-VolumeQuotaClassesTest-2004346750 tempest-VolumeQuotaClassesTest-2004346750-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2b04c8429ef46919ba62e863a0146d9', 'request_id': 'req-a22dd2d1-61f2-4acd-a3f8-441d63fdd52d'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:06.493054 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-a22dd2d1-61f2-4acd-a3f8-441d63fdd52d tempest-VolumeQuotaClassesTest-2004346750 tempest-VolumeQuotaClassesTest-2004346750-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2b04c8429ef46919ba62e863a0146d9', 'request_id': 'req-a22dd2d1-61f2-4acd-a3f8-441d63fdd52d'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:06.512454 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-a22dd2d1-61f2-4acd-a3f8-441d63fdd52d tempest-VolumeQuotaClassesTest-2004346750 tempest-VolumeQuotaClassesTest-2004346750-project-admin] The token cache is being invalidate because project f2b04c8429ef46919ba62e863a0146d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81661) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:14:06.512766 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-a22dd2d1-61f2-4acd-a3f8-441d63fdd52d tempest-VolumeQuotaClassesTest-2004346750 tempest-VolumeQuotaClassesTest-2004346750-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:06.516679 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-a22dd2d1-61f2-4acd-a3f8-441d63fdd52d tempest-VolumeQuotaClassesTest-2004346750 tempest-VolumeQuotaClassesTest-2004346750-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:06.521138 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 949/1904] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:14:06 2022] DELETE /identity/v3/projects/f2b04c8429ef46919ba62e863a0146d9 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:06.689141 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec17b60-33e3-4400-9d59-f9da23be17df tempest-VolumeQuotaClassesTest-2004346750 tempest-VolumeQuotaClassesTest-2004346750-project-admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:06.694228 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-fec17b60-33e3-4400-9d59-f9da23be17df tempest-VolumeQuotaClassesTest-2004346750 tempest-VolumeQuotaClassesTest-2004346750-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:06.818958 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec17b60-33e3-4400-9d59-f9da23be17df tempest-VolumeQuotaClassesTest-2004346750 tempest-VolumeQuotaClassesTest-2004346750-project-admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:06.851922 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22dd2d1-61f2-4acd-a3f8-441d63fdd52d tempest-VolumeQuotaClassesTest-2004346750 tempest-VolumeQuotaClassesTest-2004346750-project-admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:06.853794 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-a22dd2d1-61f2-4acd-a3f8-441d63fdd52d tempest-VolumeQuotaClassesTest-2004346750 tempest-VolumeQuotaClassesTest-2004346750-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:06.937162 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7cdc12-2b28-4acb-a01f-32f9fb2b7949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:06.938476 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7cdc12-2b28-4acb-a01f-32f9fb2b7949 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:06.938963 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7cdc12-2b28-4acb-a01f-32f9fb2b7949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:06.939517 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7cdc12-2b28-4acb-a01f-32f9fb2b7949 admin admin] PATH_INFO: `/v3/users/8635c569b0bf413aa4eb3937640e40bc` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:06.941260 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7cdc12-2b28-4acb-a01f-32f9fb2b7949 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8635c569b0bf413aa4eb3937640e40bc)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:06.943707 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7cdc12-2b28-4acb-a01f-32f9fb2b7949 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:06.987586 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22dd2d1-61f2-4acd-a3f8-441d63fdd52d tempest-VolumeQuotaClassesTest-2004346750 tempest-VolumeQuotaClassesTest-2004346750-project-admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:06.990566 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:06.996059 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07756ab1-351f-4448-8d21-6689e0d6b14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:06.996972 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07756ab1-351f-4448-8d21-6689e0d6b14a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:06.997279 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07756ab1-351f-4448-8d21-6689e0d6b14a admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:06.997767 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07756ab1-351f-4448-8d21-6689e0d6b14a admin admin] PATH_INFO: `/v3/users/a087a59950374197a928b735718d78b5` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:06.999385 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07756ab1-351f-4448-8d21-6689e0d6b14a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a087a59950374197a928b735718d78b5)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:07.002972 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07756ab1-351f-4448-8d21-6689e0d6b14a admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:07.007943 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-eb7cdc12-2b28-4acb-a01f-32f9fb2b7949 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8635c569b0bf413aa4eb3937640e40bc'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:07.024599 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-eb7cdc12-2b28-4acb-a01f-32f9fb2b7949 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8635c569b0bf413aa4eb3937640e40bc'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:07.034278 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-eb7cdc12-2b28-4acb-a01f-32f9fb2b7949 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8635c569b0bf413aa4eb3937640e40bc'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:07.043571 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-eb7cdc12-2b28-4acb-a01f-32f9fb2b7949 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8635c569b0bf413aa4eb3937640e40bc'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:07.044479 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-eb7cdc12-2b28-4acb-a01f-32f9fb2b7949 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8635c569b0bf413aa4eb3937640e40bc'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:07.055012 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 956/1905] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:14:06 2022] DELETE /identity/v3/users/8635c569b0bf413aa4eb3937640e40bc => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:07.059784 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f013a-c08b-4bdd-b717-e20639ea95c7 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:07.060303 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f013a-c08b-4bdd-b717-e20639ea95c7 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:07.060717 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f013a-c08b-4bdd-b717-e20639ea95c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:07.085976 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-c02f013a-c08b-4bdd-b717-e20639ea95c7 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:07.087465 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-c02f013a-c08b-4bdd-b717-e20639ea95c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:07.099543 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-07756ab1-351f-4448-8d21-6689e0d6b14a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a087a59950374197a928b735718d78b5'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:07.117278 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-07756ab1-351f-4448-8d21-6689e0d6b14a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a087a59950374197a928b735718d78b5'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:07.126551 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-07756ab1-351f-4448-8d21-6689e0d6b14a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a087a59950374197a928b735718d78b5'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:07.127565 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-07756ab1-351f-4448-8d21-6689e0d6b14a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a087a59950374197a928b735718d78b5'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:07.138674 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-07756ab1-351f-4448-8d21-6689e0d6b14a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a087a59950374197a928b735718d78b5'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:07.152039 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 950/1906] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:14:06 2022] DELETE /identity/v3/users/a087a59950374197a928b735718d78b5 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:07.157082 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c01a78-47d6-4040-b839-37c740bdaec6 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:07.158044 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c01a78-47d6-4040-b839-37c740bdaec6 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:07.158397 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c01a78-47d6-4040-b839-37c740bdaec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:07.186878 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-70c01a78-47d6-4040-b839-37c740bdaec6 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:07.189661 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-70c01a78-47d6-4040-b839-37c740bdaec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:07.192749 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 957/1907] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:07.205591 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:07.230265 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f013a-c08b-4bdd-b717-e20639ea95c7 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:07.231780 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-c02f013a-c08b-4bdd-b717-e20639ea95c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:07.367225 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f013a-c08b-4bdd-b717-e20639ea95c7 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:07.369953 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 951/1908] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:14:07.415204 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c01a78-47d6-4040-b839-37c740bdaec6 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:07.473813 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f85868-83ae-4e36-929e-a3c846eccaab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:07.474455 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f85868-83ae-4e36-929e-a3c846eccaab service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:07.474736 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f85868-83ae-4e36-929e-a3c846eccaab service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:07.476422 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f85868-83ae-4e36-929e-a3c846eccaab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:07.500330 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f85868-83ae-4e36-929e-a3c846eccaab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:07.502269 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f85868-83ae-4e36-929e-a3c846eccaab service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:07.502572 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c01a78-47d6-4040-b839-37c740bdaec6 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:07.506548 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f47e14-c27f-4110-b289-0fb44da8869a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:07.507174 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f47e14-c27f-4110-b289-0fb44da8869a service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:07.507496 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f47e14-c27f-4110-b289-0fb44da8869a service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:07.507767 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f47e14-c27f-4110-b289-0fb44da8869a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:07.529871 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 958/1909] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:14:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 01:14:07.533332 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f47e14-c27f-4110-b289-0fb44da8869a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:07.535186 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f47e14-c27f-4110-b289-0fb44da8869a service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:07.549157 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 952/1910] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:14:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 01:14:08.316165 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb2ec09-a5db-4f2a-9b9e-5bd93967d51c None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:08.316649 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb2ec09-a5db-4f2a-9b9e-5bd93967d51c None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:08.317742 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb2ec09-a5db-4f2a-9b9e-5bd93967d51c None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:08.324178 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 959/1911] 10.222.0.42 () {54 vars in 921 bytes} [Thu Sep 8 01:14:08 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:14:08.342258 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc404b41-2df4-4727-ba39-16f99a634cff None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:08.342708 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc404b41-2df4-4727-ba39-16f99a634cff None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:08.343766 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc404b41-2df4-4727-ba39-16f99a634cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:08.348215 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb2ec09-a5db-4f2a-9b9e-5bd93967d51c None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:08.358107 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-8fb2ec09-a5db-4f2a-9b9e-5bd93967d51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:08.394376 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-cc404b41-2df4-4727-ba39-16f99a634cff None None] MFA Rules not processed for user `ae643cfdcb164aa9b474747cd21aa116`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:08.395610 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-cc404b41-2df4-4727-ba39-16f99a634cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:08.403824 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb2ec09-a5db-4f2a-9b9e-5bd93967d51c None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:08.413515 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e9775b-bbd9-4a1a-b74c-b5e505a9417b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:08.414447 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e9775b-bbd9-4a1a-b74c-b5e505a9417b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:08.414913 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e9775b-bbd9-4a1a-b74c-b5e505a9417b admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:08.415736 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e9775b-bbd9-4a1a-b74c-b5e505a9417b admin admin] PATH_INFO: `/v3/projects/0ed48d92010942a882652234d1cb7f59` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:08.418727 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e9775b-bbd9-4a1a-b74c-b5e505a9417b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ed48d92010942a882652234d1cb7f59)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:08.425143 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e9775b-bbd9-4a1a-b74c-b5e505a9417b admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:08.430746 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:08.500086 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-25e9775b-bbd9-4a1a-b74c-b5e505a9417b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ed48d92010942a882652234d1cb7f59', 'request_id': 'req-25e9775b-bbd9-4a1a-b74c-b5e505a9417b'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:08.508389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-25e9775b-bbd9-4a1a-b74c-b5e505a9417b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ed48d92010942a882652234d1cb7f59', 'request_id': 'req-25e9775b-bbd9-4a1a-b74c-b5e505a9417b'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:08.520122 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-25e9775b-bbd9-4a1a-b74c-b5e505a9417b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ed48d92010942a882652234d1cb7f59', 'request_id': 'req-25e9775b-bbd9-4a1a-b74c-b5e505a9417b'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:08.530488 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 953/1912] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Sep 8 01:14:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:08.532033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc404b41-2df4-4727-ba39-16f99a634cff None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:08.533730 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-cc404b41-2df4-4727-ba39-16f99a634cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:08.541844 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-25e9775b-bbd9-4a1a-b74c-b5e505a9417b admin admin] The token cache is being invalidate because project 0ed48d92010942a882652234d1cb7f59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:14:08.542311 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-25e9775b-bbd9-4a1a-b74c-b5e505a9417b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:08.543166 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-25e9775b-bbd9-4a1a-b74c-b5e505a9417b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:08.549179 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 960/1913] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:14:08 2022] DELETE /identity/v3/projects/0ed48d92010942a882652234d1cb7f59 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:08.551155 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e9775b-bbd9-4a1a-b74c-b5e505a9417b admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:08.553354 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-25e9775b-bbd9-4a1a-b74c-b5e505a9417b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:08.558138 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc404b41-2df4-4727-ba39-16f99a634cff None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:08.645865 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:08.668111 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585fb544-fe2c-48a9-af90-39a4c9e54126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:08.669612 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585fb544-fe2c-48a9-af90-39a4c9e54126 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:08.671380 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585fb544-fe2c-48a9-af90-39a4c9e54126 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:08.671844 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585fb544-fe2c-48a9-af90-39a4c9e54126 admin admin] PATH_INFO: `/v3/projects/dccb74c6c96b4964a3df57f52a1f8e71` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:08.674683 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585fb544-fe2c-48a9-af90-39a4c9e54126 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dccb74c6c96b4964a3df57f52a1f8e71)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:08.676821 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585fb544-fe2c-48a9-af90-39a4c9e54126 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:08.704308 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e9775b-bbd9-4a1a-b74c-b5e505a9417b admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:08.720980 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e2bc6b-a9cd-4c24-9301-bc0f3310671b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:08.721986 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e2bc6b-a9cd-4c24-9301-bc0f3310671b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:08.722456 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e2bc6b-a9cd-4c24-9301-bc0f3310671b admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:08.722897 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e2bc6b-a9cd-4c24-9301-bc0f3310671b admin admin] PATH_INFO: `/v3/users/1b823e57eab54a9ea1e0f21e94931a73` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:08.725634 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e2bc6b-a9cd-4c24-9301-bc0f3310671b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b823e57eab54a9ea1e0f21e94931a73)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:08.729408 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e2bc6b-a9cd-4c24-9301-bc0f3310671b admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:08.791016 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-585fb544-fe2c-48a9-af90-39a4c9e54126 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dccb74c6c96b4964a3df57f52a1f8e71', 'request_id': 'req-585fb544-fe2c-48a9-af90-39a4c9e54126'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:08.805732 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-585fb544-fe2c-48a9-af90-39a4c9e54126 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dccb74c6c96b4964a3df57f52a1f8e71', 'request_id': 'req-585fb544-fe2c-48a9-af90-39a4c9e54126'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:08.818223 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-09e2bc6b-a9cd-4c24-9301-bc0f3310671b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b823e57eab54a9ea1e0f21e94931a73'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:08.826209 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-585fb544-fe2c-48a9-af90-39a4c9e54126 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dccb74c6c96b4964a3df57f52a1f8e71', 'request_id': 'req-585fb544-fe2c-48a9-af90-39a4c9e54126'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:08.848431 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-09e2bc6b-a9cd-4c24-9301-bc0f3310671b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b823e57eab54a9ea1e0f21e94931a73'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:08.852521 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-585fb544-fe2c-48a9-af90-39a4c9e54126 admin admin] The token cache is being invalidate because project dccb74c6c96b4964a3df57f52a1f8e71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81661) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:14:08.852918 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-585fb544-fe2c-48a9-af90-39a4c9e54126 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:08.853823 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-585fb544-fe2c-48a9-af90-39a4c9e54126 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:08.859555 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 954/1914] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:14:08 2022] DELETE /identity/v3/projects/dccb74c6c96b4964a3df57f52a1f8e71 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:08.859555 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585fb544-fe2c-48a9-af90-39a4c9e54126 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:08.861328 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-585fb544-fe2c-48a9-af90-39a4c9e54126 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:08.862682 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-09e2bc6b-a9cd-4c24-9301-bc0f3310671b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b823e57eab54a9ea1e0f21e94931a73'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:08.878667 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-09e2bc6b-a9cd-4c24-9301-bc0f3310671b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b823e57eab54a9ea1e0f21e94931a73'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:08.880198 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-09e2bc6b-a9cd-4c24-9301-bc0f3310671b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b823e57eab54a9ea1e0f21e94931a73'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:08.899914 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 961/1915] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:14:08 2022] DELETE /identity/v3/users/1b823e57eab54a9ea1e0f21e94931a73 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:08.901085 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e2bc6b-a9cd-4c24-9301-bc0f3310671b admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:08.905970 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-09e2bc6b-a9cd-4c24-9301-bc0f3310671b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:08.929429 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:09.017515 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585fb544-fe2c-48a9-af90-39a4c9e54126 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:09.040160 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e2bc6b-a9cd-4c24-9301-bc0f3310671b admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:09.047617 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120384e8-aa51-463b-8284-05028291dca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:09.048816 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120384e8-aa51-463b-8284-05028291dca2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:09.049379 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120384e8-aa51-463b-8284-05028291dca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:09.051275 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120384e8-aa51-463b-8284-05028291dca2 admin admin] PATH_INFO: `/v3/users/e6aa19872bd3486da5c81caeec578861` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:09.052782 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120384e8-aa51-463b-8284-05028291dca2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6aa19872bd3486da5c81caeec578861)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:09.055161 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120384e8-aa51-463b-8284-05028291dca2 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:09.116276 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-120384e8-aa51-463b-8284-05028291dca2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6aa19872bd3486da5c81caeec578861'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:09.133780 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-120384e8-aa51-463b-8284-05028291dca2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6aa19872bd3486da5c81caeec578861'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:09.153581 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-120384e8-aa51-463b-8284-05028291dca2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6aa19872bd3486da5c81caeec578861'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:09.161168 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ec387c-57f3-4a82-bbce-6af58cfb5ab2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae643cfdcb164aa9b474747cd21aa116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:09.162254 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec387c-57f3-4a82-bbce-6af58cfb5ab2 service nova] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:09.162723 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec387c-57f3-4a82-bbce-6af58cfb5ab2 service nova] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:09.163605 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec387c-57f3-4a82-bbce-6af58cfb5ab2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:09.165565 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-84ec387c-57f3-4a82-bbce-6af58cfb5ab2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:09.165941 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-120384e8-aa51-463b-8284-05028291dca2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6aa19872bd3486da5c81caeec578861'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:09.166683 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-120384e8-aa51-463b-8284-05028291dca2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6aa19872bd3486da5c81caeec578861'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:09.176220 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 962/1916] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:14:08 2022] DELETE /identity/v3/users/e6aa19872bd3486da5c81caeec578861 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:09.177184 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120384e8-aa51-463b-8284-05028291dca2 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:09.178675 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-120384e8-aa51-463b-8284-05028291dca2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:09.180796 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ec387c-57f3-4a82-bbce-6af58cfb5ab2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:09.182443 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ec387c-57f3-4a82-bbce-6af58cfb5ab2 service nova] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:09.196203 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 955/1917] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:14:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:14:09.297628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120384e8-aa51-463b-8284-05028291dca2 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:09.304895 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c29f381-619b-4af0-8d95-838abd8165e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:09.305790 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c29f381-619b-4af0-8d95-838abd8165e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:09.306386 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c29f381-619b-4af0-8d95-838abd8165e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:09.307207 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c29f381-619b-4af0-8d95-838abd8165e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:09.309704 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-3c29f381-619b-4af0-8d95-838abd8165e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:09.444559 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:09.505026 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c29f381-619b-4af0-8d95-838abd8165e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:09.508951 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c29f381-619b-4af0-8d95-838abd8165e4 service cinder] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:09.525882 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 963/1918] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:14:09 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:14:09.535579 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ec387c-57f3-4a82-bbce-6af58cfb5ab2 service nova] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:09.538137 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-84ec387c-57f3-4a82-bbce-6af58cfb5ab2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:09.552397 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:09.564224 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ec387c-57f3-4a82-bbce-6af58cfb5ab2 service nova] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:09.649053 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c29f381-619b-4af0-8d95-838abd8165e4 service cinder] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:09.650576 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-3c29f381-619b-4af0-8d95-838abd8165e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:09.764675 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f17658-844b-41f4-a264-2ed9300f123b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:09.765638 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f17658-844b-41f4-a264-2ed9300f123b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:09.765935 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f17658-844b-41f4-a264-2ed9300f123b admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:09.766217 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f17658-844b-41f4-a264-2ed9300f123b admin admin] PATH_INFO: `/v3/projects/d008c6a4bc8b4b8d98719bb402872a31` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:09.769244 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f17658-844b-41f4-a264-2ed9300f123b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d008c6a4bc8b4b8d98719bb402872a31)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:09.776057 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f17658-844b-41f4-a264-2ed9300f123b admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:09.790440 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c29f381-619b-4af0-8d95-838abd8165e4 service cinder] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:09.797104 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2094697a-471c-46c0-bd20-6d2d09346777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:09.797967 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2094697a-471c-46c0-bd20-6d2d09346777 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:09.798253 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2094697a-471c-46c0-bd20-6d2d09346777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:09.798538 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2094697a-471c-46c0-bd20-6d2d09346777 admin admin] PATH_INFO: `/v3/projects/892fe3caf9634fb78a40188f558be562` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:09.799871 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2094697a-471c-46c0-bd20-6d2d09346777 admin admin] RBAC: Authorizing `identity:delete_project(project_id=892fe3caf9634fb78a40188f558be562)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:09.802247 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2094697a-471c-46c0-bd20-6d2d09346777 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:09.887489 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-97f17658-844b-41f4-a264-2ed9300f123b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd008c6a4bc8b4b8d98719bb402872a31', 'request_id': 'req-97f17658-844b-41f4-a264-2ed9300f123b'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:09.892628 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-2094697a-471c-46c0-bd20-6d2d09346777 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '892fe3caf9634fb78a40188f558be562', 'request_id': 'req-2094697a-471c-46c0-bd20-6d2d09346777'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:09.904323 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-2094697a-471c-46c0-bd20-6d2d09346777 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '892fe3caf9634fb78a40188f558be562', 'request_id': 'req-2094697a-471c-46c0-bd20-6d2d09346777'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:09.907070 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-97f17658-844b-41f4-a264-2ed9300f123b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd008c6a4bc8b4b8d98719bb402872a31', 'request_id': 'req-97f17658-844b-41f4-a264-2ed9300f123b'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:09.913671 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-2094697a-471c-46c0-bd20-6d2d09346777 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '892fe3caf9634fb78a40188f558be562', 'request_id': 'req-2094697a-471c-46c0-bd20-6d2d09346777'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:09.927940 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-97f17658-844b-41f4-a264-2ed9300f123b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd008c6a4bc8b4b8d98719bb402872a31', 'request_id': 'req-97f17658-844b-41f4-a264-2ed9300f123b'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:09.932326 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-2094697a-471c-46c0-bd20-6d2d09346777 admin admin] The token cache is being invalidate because project 892fe3caf9634fb78a40188f558be562 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:14:09.932853 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-2094697a-471c-46c0-bd20-6d2d09346777 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:09.933860 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-2094697a-471c-46c0-bd20-6d2d09346777 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:09.936528 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 964/1919] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:14:09 2022] DELETE /identity/v3/projects/892fe3caf9634fb78a40188f558be562 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:09.943612 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff843b0-b5f4-472c-b6fe-291a6187dfce None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:09.945342 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff843b0-b5f4-472c-b6fe-291a6187dfce None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:09.946455 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff843b0-b5f4-472c-b6fe-291a6187dfce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:09.951729 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-97f17658-844b-41f4-a264-2ed9300f123b admin admin] The token cache is being invalidate because project d008c6a4bc8b4b8d98719bb402872a31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81661) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:14:09.952114 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-97f17658-844b-41f4-a264-2ed9300f123b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:09.953728 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-97f17658-844b-41f4-a264-2ed9300f123b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:09.960050 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 956/1920] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:14:09 2022] DELETE /identity/v3/projects/d008c6a4bc8b4b8d98719bb402872a31 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:09.971355 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa73c8-7024-4546-87b3-31d902a68e03 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:09.971739 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa73c8-7024-4546-87b3-31d902a68e03 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:09.972092 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa73c8-7024-4546-87b3-31d902a68e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:09.994245 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-dff843b0-b5f4-472c-b6fe-291a6187dfce None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:09.996272 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-dff843b0-b5f4-472c-b6fe-291a6187dfce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:09.999478 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-17aa73c8-7024-4546-87b3-31d902a68e03 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:10.001060 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-17aa73c8-7024-4546-87b3-31d902a68e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:10.111705 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 965/1921] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:10.119409 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87547a5-80b3-42e8-b308-3f2c21e8ccc7 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:10.119815 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87547a5-80b3-42e8-b308-3f2c21e8ccc7 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:10.120261 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87547a5-80b3-42e8-b308-3f2c21e8ccc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:10.165731 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 957/1922] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:10.173476 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb353e3d-9780-43de-93f4-ed212985cf28 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:10.173766 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb353e3d-9780-43de-93f4-ed212985cf28 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:10.174044 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb353e3d-9780-43de-93f4-ed212985cf28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:10.188070 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-e87547a5-80b3-42e8-b308-3f2c21e8ccc7 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:10.190443 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-e87547a5-80b3-42e8-b308-3f2c21e8ccc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:10.224797 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-fb353e3d-9780-43de-93f4-ed212985cf28 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:10.225670 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 966/1923] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:14:10.229633 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-fb353e3d-9780-43de-93f4-ed212985cf28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:10.230056 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87547a5-80b3-42e8-b308-3f2c21e8ccc7 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:10.243524 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:10.245527 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 958/1924] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:10.250409 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb353e3d-9780-43de-93f4-ed212985cf28 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:10.252573 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87547a5-80b3-42e8-b308-3f2c21e8ccc7 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:10.267441 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0030fe43-8e9f-4dca-bee8-53c3eb322ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:10.268085 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030fe43-8e9f-4dca-bee8-53c3eb322ddd admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:10.268388 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030fe43-8e9f-4dca-bee8-53c3eb322ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:10.268663 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030fe43-8e9f-4dca-bee8-53c3eb322ddd admin admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:10.269345 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0030fe43-8e9f-4dca-bee8-53c3eb322ddd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:14:10.269625 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0030fe43-8e9f-4dca-bee8-53c3eb322ddd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:10.272423 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb353e3d-9780-43de-93f4-ed212985cf28 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:10.282292 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0030fe43-8e9f-4dca-bee8-53c3eb322ddd admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:10.284388 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce136419-b2e9-4201-8f0a-37c1515db6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:10.296041 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce136419-b2e9-4201-8f0a-37c1515db6e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:10.297406 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce136419-b2e9-4201-8f0a-37c1515db6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:10.297727 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce136419-b2e9-4201-8f0a-37c1515db6e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:10.298544 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce136419-b2e9-4201-8f0a-37c1515db6e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:14:10.298856 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce136419-b2e9-4201-8f0a-37c1515db6e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:10.300611 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce136419-b2e9-4201-8f0a-37c1515db6e7 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:10.304457 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 967/1925] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:14:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:14:10.309410 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0030fe43-8e9f-4dca-bee8-53c3eb322ddd admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:10.320181 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 959/1926] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:14:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:14:10.324612 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce136419-b2e9-4201-8f0a-37c1515db6e7 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:10.328002 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0030fe43-8e9f-4dca-bee8-53c3eb322ddd admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:10.335534 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e6b3eb-2269-4049-ae78-61aa4dedc45e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:10.336905 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e6b3eb-2269-4049-ae78-61aa4dedc45e admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:10.337508 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e6b3eb-2269-4049-ae78-61aa4dedc45e admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:10.338104 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e6b3eb-2269-4049-ae78-61aa4dedc45e admin admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:10.339187 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e6b3eb-2269-4049-ae78-61aa4dedc45e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:10.341340 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce136419-b2e9-4201-8f0a-37c1515db6e7 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:10.342031 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e6b3eb-2269-4049-ae78-61aa4dedc45e admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:10.357182 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da817d9-cfbd-457b-9907-1c73609314ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:10.357890 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da817d9-cfbd-457b-9907-1c73609314ba admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:10.358053 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da817d9-cfbd-457b-9907-1c73609314ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:10.358331 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da817d9-cfbd-457b-9907-1c73609314ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:10.359367 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da817d9-cfbd-457b-9907-1c73609314ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:10.361618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da817d9-cfbd-457b-9907-1c73609314ba admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:10.392248 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 968/1927] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:14:10 2022] POST /identity/v3/projects => generated 379 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:14:10.393992 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e6b3eb-2269-4049-ae78-61aa4dedc45e admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:10.397074 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:10.419368 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 960/1928] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:14:10 2022] POST /identity/v3/projects => generated 371 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:14:10.424407 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da817d9-cfbd-457b-9907-1c73609314ba admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:10.523219 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da817d9-cfbd-457b-9907-1c73609314ba admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:10.530043 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31042b3-d08f-40de-8a15-d8eb2e86c625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:10.532236 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31042b3-d08f-40de-8a15-d8eb2e86c625 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:10.532787 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31042b3-d08f-40de-8a15-d8eb2e86c625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:10.533513 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31042b3-d08f-40de-8a15-d8eb2e86c625 admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:10.534611 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31042b3-d08f-40de-8a15-d8eb2e86c625 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:10.536996 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31042b3-d08f-40de-8a15-d8eb2e86c625 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:10.537549 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e6b3eb-2269-4049-ae78-61aa4dedc45e admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:10.544701 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33357aa8-244e-4a22-bcac-621c8afcecdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:10.545512 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33357aa8-244e-4a22-bcac-621c8afcecdd admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:10.546523 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33357aa8-244e-4a22-bcac-621c8afcecdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:10.546637 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33357aa8-244e-4a22-bcac-621c8afcecdd admin admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:10.547400 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33357aa8-244e-4a22-bcac-621c8afcecdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:10.555539 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33357aa8-244e-4a22-bcac-621c8afcecdd admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:10.575069 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 961/1929] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:14:10 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:14:10.580441 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31042b3-d08f-40de-8a15-d8eb2e86c625 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:10.594197 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 969/1930] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:14:10 2022] POST /identity/v3/users => generated 290 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:14:10.596633 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31042b3-d08f-40de-8a15-d8eb2e86c625 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:10.598544 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33357aa8-244e-4a22-bcac-621c8afcecdd admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:10.602406 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a22904-d45a-4d91-81c8-34292fc14e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:10.603135 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a22904-d45a-4d91-81c8-34292fc14e27 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:10.603514 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a22904-d45a-4d91-81c8-34292fc14e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:10.603884 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a22904-d45a-4d91-81c8-34292fc14e27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:10.604907 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a22904-d45a-4d91-81c8-34292fc14e27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:10.606707 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a22904-d45a-4d91-81c8-34292fc14e27 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:10.618045 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33357aa8-244e-4a22-bcac-621c8afcecdd admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:10.621754 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 962/1931] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:14:10 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:14:10.622990 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a22904-d45a-4d91-81c8-34292fc14e27 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:10.628006 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90529e2-a538-4f3f-abb4-4d4a6a670188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:10.628869 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90529e2-a538-4f3f-abb4-4d4a6a670188 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:10.629293 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90529e2-a538-4f3f-abb4-4d4a6a670188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:10.629707 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90529e2-a538-4f3f-abb4-4d4a6a670188 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:10.630495 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90529e2-a538-4f3f-abb4-4d4a6a670188 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:10.632563 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90529e2-a538-4f3f-abb4-4d4a6a670188 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:10.637999 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a22904-d45a-4d91-81c8-34292fc14e27 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:10.644358 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf415ff-bdde-4081-a9bf-c9b4e27837da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:10.645511 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf415ff-bdde-4081-a9bf-c9b4e27837da admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:10.645943 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf415ff-bdde-4081-a9bf-c9b4e27837da admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:10.646316 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf415ff-bdde-4081-a9bf-c9b4e27837da admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:10.647089 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf415ff-bdde-4081-a9bf-c9b4e27837da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:10.648867 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf415ff-bdde-4081-a9bf-c9b4e27837da admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:10.656338 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 970/1932] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:14:10 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:14:10.658415 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90529e2-a538-4f3f-abb4-4d4a6a670188 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:10.667772 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 963/1933] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:14:10 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:14:10.669219 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf415ff-bdde-4081-a9bf-c9b4e27837da admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:10.674391 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90529e2-a538-4f3f-abb4-4d4a6a670188 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:10.680304 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cd78dd-d483-4629-8291-38c90ab2d988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:10.681201 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd78dd-d483-4629-8291-38c90ab2d988 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:10.681626 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd78dd-d483-4629-8291-38c90ab2d988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:10.682049 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd78dd-d483-4629-8291-38c90ab2d988 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:10.682854 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf415ff-bdde-4081-a9bf-c9b4e27837da admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:10.682971 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cd78dd-d483-4629-8291-38c90ab2d988 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:10.685925 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cd78dd-d483-4629-8291-38c90ab2d988 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:10.693421 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176575bc-3ac9-48e2-8537-c3279d81fda4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:10.694849 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176575bc-3ac9-48e2-8537-c3279d81fda4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:10.695254 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176575bc-3ac9-48e2-8537-c3279d81fda4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:10.695632 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176575bc-3ac9-48e2-8537-c3279d81fda4 admin admin] PATH_INFO: `/v3/projects/98839645cd3c4f5eb0339cb6c6304d2d/users/a833d3dfb459415886b8798cd3800237/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:10.707735 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 971/1934] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:14:10 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:14:10.709175 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cd78dd-d483-4629-8291-38c90ab2d988 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:10.722164 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cd78dd-d483-4629-8291-38c90ab2d988 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:10.727694 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883957bc-c682-47d1-b5e5-a35def240374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:10.729388 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883957bc-c682-47d1-b5e5-a35def240374 admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:10.729714 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883957bc-c682-47d1-b5e5-a35def240374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:10.729993 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883957bc-c682-47d1-b5e5-a35def240374 admin admin] PATH_INFO: `/v3/projects/d17cfb790998450895f07969087655e9/users/e6169c2629724401b68c70e7b79b621f/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:10.733151 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176575bc-3ac9-48e2-8537-c3279d81fda4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98839645cd3c4f5eb0339cb6c6304d2d, user_id=a833d3dfb459415886b8798cd3800237, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:10.738773 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176575bc-3ac9-48e2-8537-c3279d81fda4 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:10.755837 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883957bc-c682-47d1-b5e5-a35def240374 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d17cfb790998450895f07969087655e9, user_id=e6169c2629724401b68c70e7b79b621f, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:10.757492 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 964/1935] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:14:10 2022] PUT /identity/v3/projects/98839645cd3c4f5eb0339cb6c6304d2d/users/a833d3dfb459415886b8798cd3800237/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:10.762759 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c94d3b-72ac-47b6-9d91-d385aa9090e8 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:10.763306 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c94d3b-72ac-47b6-9d91-d385aa9090e8 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:10.763417 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883957bc-c682-47d1-b5e5-a35def240374 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:10.764001 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c94d3b-72ac-47b6-9d91-d385aa9090e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:10.785716 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 972/1936] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:14:10 2022] PUT /identity/v3/projects/d17cfb790998450895f07969087655e9/users/e6169c2629724401b68c70e7b79b621f/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:10.789683 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-d7c94d3b-72ac-47b6-9d91-d385aa9090e8 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:10.791277 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-d7c94d3b-72ac-47b6-9d91-d385aa9090e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:10.791576 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f4bf4-b7e1-465a-8793-695f7707956b None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:10.791869 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f4bf4-b7e1-465a-8793-695f7707956b None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:10.792161 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f4bf4-b7e1-465a-8793-695f7707956b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:10.826551 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-f83f4bf4-b7e1-465a-8793-695f7707956b None None] MFA Rules not processed for user `e6169c2629724401b68c70e7b79b621f`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:10.827804 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-f83f4bf4-b7e1-465a-8793-695f7707956b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:10.856398 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 965/1937] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:10.869918 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c94d3b-72ac-47b6-9d91-d385aa9090e8 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:10.871554 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-d7c94d3b-72ac-47b6-9d91-d385aa9090e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:10.922808 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 973/1938] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:14:10.926869 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f4bf4-b7e1-465a-8793-695f7707956b None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:10.936177 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:10.946339 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:10.952074 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f4bf4-b7e1-465a-8793-695f7707956b None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:10.958736 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97b855b-f15d-43f8-b532-ff0bfa8032b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:10.959454 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97b855b-f15d-43f8-b532-ff0bfa8032b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:10.959717 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97b855b-f15d-43f8-b532-ff0bfa8032b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:10.959993 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97b855b-f15d-43f8-b532-ff0bfa8032b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:10.960671 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97b855b-f15d-43f8-b532-ff0bfa8032b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:10.963562 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97b855b-f15d-43f8-b532-ff0bfa8032b5 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:10.977279 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c94d3b-72ac-47b6-9d91-d385aa9090e8 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:10.987227 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433bc097-e304-4e13-aefe-a462afcf1373 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:10.988365 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433bc097-e304-4e13-aefe-a462afcf1373 service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:10.988733 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433bc097-e304-4e13-aefe-a462afcf1373 service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:10.989226 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433bc097-e304-4e13-aefe-a462afcf1373 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:11.001603 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433bc097-e304-4e13-aefe-a462afcf1373 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:11.003447 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433bc097-e304-4e13-aefe-a462afcf1373 service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:11.018263 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 966/1939] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:14:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 01:14:11.029462 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 974/1940] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:14:10 2022] POST /identity/v3/projects => generated 379 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:14:11.034737 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433bc097-e304-4e13-aefe-a462afcf1373 service neutron] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:11.047788 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433bc097-e304-4e13-aefe-a462afcf1373 service neutron] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:11.114672 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f791f1f9-075d-43c8-8848-efcb465a211e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:11.115511 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791f1f9-075d-43c8-8848-efcb465a211e admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:11.115857 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791f1f9-075d-43c8-8848-efcb465a211e admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:11.116231 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791f1f9-075d-43c8-8848-efcb465a211e admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:11.116939 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f791f1f9-075d-43c8-8848-efcb465a211e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:11.119081 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f791f1f9-075d-43c8-8848-efcb465a211e admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:11.141329 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 967/1941] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:14:11 2022] POST /identity/v3/users => generated 290 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:14:11.145655 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97b855b-f15d-43f8-b532-ff0bfa8032b5 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:11.157331 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97b855b-f15d-43f8-b532-ff0bfa8032b5 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:11.162754 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3143445-9413-4f38-a7ad-59dc45d80721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:11.163580 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3143445-9413-4f38-a7ad-59dc45d80721 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:11.163928 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3143445-9413-4f38-a7ad-59dc45d80721 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:11.164328 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3143445-9413-4f38-a7ad-59dc45d80721 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:11.165195 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3143445-9413-4f38-a7ad-59dc45d80721 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:11.166940 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3143445-9413-4f38-a7ad-59dc45d80721 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:11.178562 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 975/1942] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:14:11.182528 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f791f1f9-075d-43c8-8848-efcb465a211e admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:11.193939 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f791f1f9-075d-43c8-8848-efcb465a211e admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:11.201065 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5959ea57-d6fc-4c0d-a5b9-c4faef4189ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:11.203092 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5959ea57-d6fc-4c0d-a5b9-c4faef4189ec admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:11.203092 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5959ea57-d6fc-4c0d-a5b9-c4faef4189ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:11.203092 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5959ea57-d6fc-4c0d-a5b9-c4faef4189ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:11.203544 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5959ea57-d6fc-4c0d-a5b9-c4faef4189ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:11.205453 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5959ea57-d6fc-4c0d-a5b9-c4faef4189ec admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:11.219794 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 968/1943] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:14:11.224070 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3143445-9413-4f38-a7ad-59dc45d80721 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:11.239357 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3143445-9413-4f38-a7ad-59dc45d80721 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:11.247483 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd9baf-6f67-4f83-ba8b-3b51af869963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:11.248640 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd9baf-6f67-4f83-ba8b-3b51af869963 admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:11.260776 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd9baf-6f67-4f83-ba8b-3b51af869963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:11.261341 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd9baf-6f67-4f83-ba8b-3b51af869963 admin admin] PATH_INFO: `/v3/projects/a760b6850ecf45938c0ee87e2e5f30e7/users/f117d4d42ea24fb5a0116338ab988390/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:11.297968 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dd9baf-6f67-4f83-ba8b-3b51af869963 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a760b6850ecf45938c0ee87e2e5f30e7, user_id=f117d4d42ea24fb5a0116338ab988390, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:11.301375 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dd9baf-6f67-4f83-ba8b-3b51af869963 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:11.329793 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 976/1944] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:14:11 2022] PUT /identity/v3/projects/a760b6850ecf45938c0ee87e2e5f30e7/users/f117d4d42ea24fb5a0116338ab988390/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:11.341528 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb58cea-e506-4564-a352-6f6063d50fa1 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:11.341817 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb58cea-e506-4564-a352-6f6063d50fa1 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:11.342089 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb58cea-e506-4564-a352-6f6063d50fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:11.399033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-8bb58cea-e506-4564-a352-6f6063d50fa1 None None] MFA Rules not processed for user `f117d4d42ea24fb5a0116338ab988390`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:11.400486 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-8bb58cea-e506-4564-a352-6f6063d50fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:11.498812 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 969/1945] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:11.503803 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd9baf-6f67-4f83-ba8b-3b51af869963 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:11.663125 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd9baf-6f67-4f83-ba8b-3b51af869963 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:11.668958 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10da32f-1f37-469a-996e-04c869f6319a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:11.669964 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10da32f-1f37-469a-996e-04c869f6319a admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:11.669964 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10da32f-1f37-469a-996e-04c869f6319a admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:11.673645 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10da32f-1f37-469a-996e-04c869f6319a admin admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:11.674597 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10da32f-1f37-469a-996e-04c869f6319a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:11.677563 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10da32f-1f37-469a-996e-04c869f6319a admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:11.732188 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 977/1946] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:14:11 2022] POST /identity/v3/projects => generated 379 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:14:11.735770 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb58cea-e506-4564-a352-6f6063d50fa1 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:11.748181 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb58cea-e506-4564-a352-6f6063d50fa1 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:11.830414 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:11.845732 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2389728-23ac-408d-bab0-5b994dbd2ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:11.846479 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2389728-23ac-408d-bab0-5b994dbd2ae0 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:11.846830 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2389728-23ac-408d-bab0-5b994dbd2ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:11.847150 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2389728-23ac-408d-bab0-5b994dbd2ae0 admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:11.847802 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2389728-23ac-408d-bab0-5b994dbd2ae0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:11.850236 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2389728-23ac-408d-bab0-5b994dbd2ae0 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:11.882290 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 970/1947] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:14:11 2022] POST /identity/v3/users => generated 296 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:14:11.883659 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10da32f-1f37-469a-996e-04c869f6319a admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:11.897205 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10da32f-1f37-469a-996e-04c869f6319a admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:11.904892 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5dc659-620b-4aae-9d4a-f2172ede6ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:11.907998 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5dc659-620b-4aae-9d4a-f2172ede6ff1 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:11.908571 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5dc659-620b-4aae-9d4a-f2172ede6ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:11.909424 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5dc659-620b-4aae-9d4a-f2172ede6ff1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:11.910764 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5dc659-620b-4aae-9d4a-f2172ede6ff1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:11.913193 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5dc659-620b-4aae-9d4a-f2172ede6ff1 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:11.927733 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 978/1948] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:14:11.931903 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2389728-23ac-408d-bab0-5b994dbd2ae0 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:11.943400 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2389728-23ac-408d-bab0-5b994dbd2ae0 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:11.948606 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceeeecb-34ee-4054-b90c-7507b94dc9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:11.949317 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceeeecb-34ee-4054-b90c-7507b94dc9a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:11.949674 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceeeecb-34ee-4054-b90c-7507b94dc9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:11.950354 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceeeecb-34ee-4054-b90c-7507b94dc9a3 admin admin] PATH_INFO: `/v3/projects/d986d13fb1c44eebb563871b2ef748e7/users/126eae716681430b8a08c798ef218b68/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:11.990220 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceeeecb-34ee-4054-b90c-7507b94dc9a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d986d13fb1c44eebb563871b2ef748e7, user_id=126eae716681430b8a08c798ef218b68, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:11.993864 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceeeecb-34ee-4054-b90c-7507b94dc9a3 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:12.016125 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 971/1949] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:14:11 2022] PUT /identity/v3/projects/d986d13fb1c44eebb563871b2ef748e7/users/126eae716681430b8a08c798ef218b68/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:12.017056 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5dc659-620b-4aae-9d4a-f2172ede6ff1 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:12.028596 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5dc659-620b-4aae-9d4a-f2172ede6ff1 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:12.118023 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87351adb-3920-44d0-b201-1e0ac874ac69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:12.118775 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87351adb-3920-44d0-b201-1e0ac874ac69 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:12.119159 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87351adb-3920-44d0-b201-1e0ac874ac69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:12.119538 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87351adb-3920-44d0-b201-1e0ac874ac69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:12.120736 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87351adb-3920-44d0-b201-1e0ac874ac69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:12.122073 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87351adb-3920-44d0-b201-1e0ac874ac69 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:12.136020 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 979/1950] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:14:12.139301 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceeeecb-34ee-4054-b90c-7507b94dc9a3 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:12.160550 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d5392-71ea-4c8b-a531-5f2422153757 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:12.161021 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d5392-71ea-4c8b-a531-5f2422153757 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:12.161341 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d5392-71ea-4c8b-a531-5f2422153757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:12.170676 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceeeecb-34ee-4054-b90c-7507b94dc9a3 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:12.179052 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ff9e78-4f3a-421a-964c-cd280902a477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:12.180057 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff9e78-4f3a-421a-964c-cd280902a477 admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:12.180493 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff9e78-4f3a-421a-964c-cd280902a477 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:12.180907 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff9e78-4f3a-421a-964c-cd280902a477 admin admin] PATH_INFO: `/v3/projects/d986d13fb1c44eebb563871b2ef748e7/users/126eae716681430b8a08c798ef218b68/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:12.184299 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ff9e78-4f3a-421a-964c-cd280902a477 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d986d13fb1c44eebb563871b2ef748e7, user_id=126eae716681430b8a08c798ef218b68, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:12.187905 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ff9e78-4f3a-421a-964c-cd280902a477 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:12.204837 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 972/1951] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:14:12 2022] PUT /identity/v3/projects/d986d13fb1c44eebb563871b2ef748e7/users/126eae716681430b8a08c798ef218b68/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:12.206679 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-ef1d5392-71ea-4c8b-a531-5f2422153757 None None] MFA Rules not processed for user `a833d3dfb459415886b8798cd3800237`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:12.208804 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-ef1d5392-71ea-4c8b-a531-5f2422153757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:12.215799 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:12.220181 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d823d2-c8c7-4b07-bc67-e9093604be4d None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:12.220181 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d823d2-c8c7-4b07-bc67-e9093604be4d None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:12.220181 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d823d2-c8c7-4b07-bc67-e9093604be4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:12.333941 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-62d823d2-c8c7-4b07-bc67-e9093604be4d None None] MFA Rules not processed for user `126eae716681430b8a08c798ef218b68`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:12.335162 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-62d823d2-c8c7-4b07-bc67-e9093604be4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:12.395070 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 980/1952] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:12.400071 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d5392-71ea-4c8b-a531-5f2422153757 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:12.460655 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 973/1953] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:12 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:12.470892 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d823d2-c8c7-4b07-bc67-e9093604be4d None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:12.472784 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-62d823d2-c8c7-4b07-bc67-e9093604be4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:12.541420 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d5392-71ea-4c8b-a531-5f2422153757 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:12.551103 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0fa64d-3793-4592-bc2b-f31b3b85e8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:12.551907 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0fa64d-3793-4592-bc2b-f31b3b85e8a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:12.553028 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0fa64d-3793-4592-bc2b-f31b3b85e8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:12.555040 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0fa64d-3793-4592-bc2b-f31b3b85e8a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:12.555854 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0fa64d-3793-4592-bc2b-f31b3b85e8a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:12.558590 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0fa64d-3793-4592-bc2b-f31b3b85e8a0 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:12.604761 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d823d2-c8c7-4b07-bc67-e9093604be4d None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:12.609077 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653efc5d-fdba-43ef-b6d8-a6a73c559466 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:12.610023 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653efc5d-fdba-43ef-b6d8-a6a73c559466 service cinder] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:12.610488 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653efc5d-fdba-43ef-b6d8-a6a73c559466 service cinder] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:12.611252 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653efc5d-fdba-43ef-b6d8-a6a73c559466 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:12.648698 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653efc5d-fdba-43ef-b6d8-a6a73c559466 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:12.650770 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653efc5d-fdba-43ef-b6d8-a6a73c559466 service cinder] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:12.653809 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 981/1954] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:14:12 2022] POST /identity/v3/projects => generated 371 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:14:12.659512 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0fa64d-3793-4592-bc2b-f31b3b85e8a0 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:12.671141 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 974/1955] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:14:12 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:14:12.760092 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653efc5d-fdba-43ef-b6d8-a6a73c559466 service cinder] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:12.771734 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653efc5d-fdba-43ef-b6d8-a6a73c559466 service cinder] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:12.783748 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0fa64d-3793-4592-bc2b-f31b3b85e8a0 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:12.789476 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97794d4-e003-4699-a4c4-435c8a5faf1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:12.790186 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97794d4-e003-4699-a4c4-435c8a5faf1c admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:12.790466 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97794d4-e003-4699-a4c4-435c8a5faf1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:12.790743 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97794d4-e003-4699-a4c4-435c8a5faf1c admin admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:12.791668 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97794d4-e003-4699-a4c4-435c8a5faf1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:12.793573 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97794d4-e003-4699-a4c4-435c8a5faf1c admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:12.824914 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 982/1956] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:14:12 2022] POST /identity/v3/users => generated 292 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:14:12.826305 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97794d4-e003-4699-a4c4-435c8a5faf1c admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:12.836622 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:12.851930 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97794d4-e003-4699-a4c4-435c8a5faf1c admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:12.858091 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab55a1ac-2dea-409d-b3a9-e61d4f91e0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:12.859101 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab55a1ac-2dea-409d-b3a9-e61d4f91e0f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:12.859806 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab55a1ac-2dea-409d-b3a9-e61d4f91e0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:12.860209 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab55a1ac-2dea-409d-b3a9-e61d4f91e0f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:12.861020 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab55a1ac-2dea-409d-b3a9-e61d4f91e0f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:12.864129 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab55a1ac-2dea-409d-b3a9-e61d4f91e0f9 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:12.929023 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b18f7b-a643-4c88-abd7-1b5f6c4109cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:12.929852 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b18f7b-a643-4c88-abd7-1b5f6c4109cb service cinder] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:12.929963 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 983/1957] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:14:12.930042 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b18f7b-a643-4c88-abd7-1b5f6c4109cb service cinder] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:12.930383 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b18f7b-a643-4c88-abd7-1b5f6c4109cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:12.934997 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab55a1ac-2dea-409d-b3a9-e61d4f91e0f9 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:12.949088 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab55a1ac-2dea-409d-b3a9-e61d4f91e0f9 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:12.957373 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408fb27-aa3a-4ba7-a1dc-26218baa781f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:12.958185 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408fb27-aa3a-4ba7-a1dc-26218baa781f admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:12.958559 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408fb27-aa3a-4ba7-a1dc-26218baa781f admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:12.958924 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408fb27-aa3a-4ba7-a1dc-26218baa781f admin admin] PATH_INFO: `/v3/projects/0308978cd8c142c6ad93e27155824889/users/79ac51f5326f41d1872bf70cd7bb2981/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:12.964444 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:13.006253 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3408fb27-aa3a-4ba7-a1dc-26218baa781f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0308978cd8c142c6ad93e27155824889, user_id=79ac51f5326f41d1872bf70cd7bb2981, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:13.017005 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3408fb27-aa3a-4ba7-a1dc-26218baa781f admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:13.055255 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 984/1958] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:14:12 2022] PUT /identity/v3/projects/0308978cd8c142c6ad93e27155824889/users/79ac51f5326f41d1872bf70cd7bb2981/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:13.056887 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408fb27-aa3a-4ba7-a1dc-26218baa781f admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:13.090749 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b18f7b-a643-4c88-abd7-1b5f6c4109cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:13.094882 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b18f7b-a643-4c88-abd7-1b5f6c4109cb service cinder] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:13.114968 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 975/1959] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:14:12 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 01:14:13.186517 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408fb27-aa3a-4ba7-a1dc-26218baa781f admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:13.196758 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4e4fe1-3769-4918-80ea-df46a892e9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:13.196758 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4e4fe1-3769-4918-80ea-df46a892e9a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:13.196758 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4e4fe1-3769-4918-80ea-df46a892e9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:13.196758 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4e4fe1-3769-4918-80ea-df46a892e9a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:13.197058 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4e4fe1-3769-4918-80ea-df46a892e9a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:13.197058 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4e4fe1-3769-4918-80ea-df46a892e9a7 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:13.208763 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 985/1960] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:14:13 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:14:13.213479 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b18f7b-a643-4c88-abd7-1b5f6c4109cb service cinder] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:13.226341 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b18f7b-a643-4c88-abd7-1b5f6c4109cb service cinder] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:13.231905 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcdfe92-f3b4-4b60-b00c-8eda7eb6453a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:13.232856 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcdfe92-f3b4-4b60-b00c-8eda7eb6453a admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:13.233170 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcdfe92-f3b4-4b60-b00c-8eda7eb6453a admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:13.233519 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcdfe92-f3b4-4b60-b00c-8eda7eb6453a admin admin] PATH_INFO: `/v3/projects/0308978cd8c142c6ad93e27155824889/users/79ac51f5326f41d1872bf70cd7bb2981/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:13.235984 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcdfe92-f3b4-4b60-b00c-8eda7eb6453a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0308978cd8c142c6ad93e27155824889, user_id=79ac51f5326f41d1872bf70cd7bb2981, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:13.239300 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcdfe92-f3b4-4b60-b00c-8eda7eb6453a admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:13.259726 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 976/1961] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:14:13 2022] PUT /identity/v3/projects/0308978cd8c142c6ad93e27155824889/users/79ac51f5326f41d1872bf70cd7bb2981/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:14.025003 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d504c-2475-42b3-acda-b3a442c3dba9 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:14.025837 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d504c-2475-42b3-acda-b3a442c3dba9 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:14.026275 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d504c-2475-42b3-acda-b3a442c3dba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:14.064405 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-8a5d504c-2475-42b3-acda-b3a442c3dba9 None None] MFA Rules not processed for user `79ac51f5326f41d1872bf70cd7bb2981`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:14.065842 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-8a5d504c-2475-42b3-acda-b3a442c3dba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:14.175346 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 986/1962] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:14 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:14.188860 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcdfe92-f3b4-4b60-b00c-8eda7eb6453a admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:14.191298 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-0bcdfe92-f3b4-4b60-b00c-8eda7eb6453a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:14.270253 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcdfe92-f3b4-4b60-b00c-8eda7eb6453a admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:14.273969 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83e1bc0-b8b6-408c-a2ac-cb7956aada56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:14.274590 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e1bc0-b8b6-408c-a2ac-cb7956aada56 service cinder] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:14.274868 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e1bc0-b8b6-408c-a2ac-cb7956aada56 service cinder] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:14.275140 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e1bc0-b8b6-408c-a2ac-cb7956aada56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:14.344998 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83e1bc0-b8b6-408c-a2ac-cb7956aada56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:14.346582 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83e1bc0-b8b6-408c-a2ac-cb7956aada56 service cinder] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:14.357854 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 977/1963] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:14:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:14:17.855205 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5d504c-2475-42b3-acda-b3a442c3dba9 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:17.865185 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5d504c-2475-42b3-acda-b3a442c3dba9 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:17.868877 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5692a2-7eea-4b00-bd0b-529df74984c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:17.869475 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5692a2-7eea-4b00-bd0b-529df74984c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:17.869750 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5692a2-7eea-4b00-bd0b-529df74984c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:17.870018 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5692a2-7eea-4b00-bd0b-529df74984c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:17.879937 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5692a2-7eea-4b00-bd0b-529df74984c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:17.881413 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5692a2-7eea-4b00-bd0b-529df74984c3 service cinder] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:17.891694 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 987/1964] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:14:17 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:14:18.344271 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83e1bc0-b8b6-408c-a2ac-cb7956aada56 service cinder] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:18.372256 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:18.427408 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83e1bc0-b8b6-408c-a2ac-cb7956aada56 service cinder] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:18.435802 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe0b1cb-8ae2-4218-b7c8-94c75acb1358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:18.437048 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe0b1cb-8ae2-4218-b7c8-94c75acb1358 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:18.437497 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe0b1cb-8ae2-4218-b7c8-94c75acb1358 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:18.437778 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe0b1cb-8ae2-4218-b7c8-94c75acb1358 admin admin] PATH_INFO: `/v3/users/e6169c2629724401b68c70e7b79b621f` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:18.438935 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe0b1cb-8ae2-4218-b7c8-94c75acb1358 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6169c2629724401b68c70e7b79b621f)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:18.440994 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe0b1cb-8ae2-4218-b7c8-94c75acb1358 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:18.494728 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-2fe0b1cb-8ae2-4218-b7c8-94c75acb1358 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6169c2629724401b68c70e7b79b621f'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:18.509857 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-2fe0b1cb-8ae2-4218-b7c8-94c75acb1358 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6169c2629724401b68c70e7b79b621f'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:18.519385 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-2fe0b1cb-8ae2-4218-b7c8-94c75acb1358 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6169c2629724401b68c70e7b79b621f'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:18.520157 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-2fe0b1cb-8ae2-4218-b7c8-94c75acb1358 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6169c2629724401b68c70e7b79b621f'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:18.528606 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-2fe0b1cb-8ae2-4218-b7c8-94c75acb1358 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6169c2629724401b68c70e7b79b621f'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:18.535004 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 978/1965] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:14:18 2022] DELETE /identity/v3/users/e6169c2629724401b68c70e7b79b621f => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:18.539976 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf8ab93-d260-4cc1-93c3-67bed771b2c7 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:18.540303 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf8ab93-d260-4cc1-93c3-67bed771b2c7 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:18.540587 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf8ab93-d260-4cc1-93c3-67bed771b2c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:18.562221 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-adf8ab93-d260-4cc1-93c3-67bed771b2c7 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:18.563006 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-adf8ab93-d260-4cc1-93c3-67bed771b2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:18.594061 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe0b1cb-8ae2-4218-b7c8-94c75acb1358 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:18.595931 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-2fe0b1cb-8ae2-4218-b7c8-94c75acb1358 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:18.639101 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:18.673576 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 988/1966] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:18.692941 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf8ab93-d260-4cc1-93c3-67bed771b2c7 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:18.694551 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-adf8ab93-d260-4cc1-93c3-67bed771b2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:18.730469 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe0b1cb-8ae2-4218-b7c8-94c75acb1358 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:18.736640 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c42c087-ed2a-4c61-b851-d96e14f4776e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:18.737477 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c42c087-ed2a-4c61-b851-d96e14f4776e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:18.737852 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c42c087-ed2a-4c61-b851-d96e14f4776e admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:18.738758 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c42c087-ed2a-4c61-b851-d96e14f4776e admin admin] PATH_INFO: `/v3/users/2f1c9901ebd541d18e1bb3f82ad237c0` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:18.740036 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c42c087-ed2a-4c61-b851-d96e14f4776e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f1c9901ebd541d18e1bb3f82ad237c0)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:18.742048 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c42c087-ed2a-4c61-b851-d96e14f4776e admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:18.789347 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf8ab93-d260-4cc1-93c3-67bed771b2c7 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:18.794102 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb194b4-1137-4ae8-863b-f1e8d16c1e1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:18.795558 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb194b4-1137-4ae8-863b-f1e8d16c1e1e service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:18.795880 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb194b4-1137-4ae8-863b-f1e8d16c1e1e service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:18.796281 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb194b4-1137-4ae8-863b-f1e8d16c1e1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:18.808782 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb194b4-1137-4ae8-863b-f1e8d16c1e1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:18.809248 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:18.810277 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb194b4-1137-4ae8-863b-f1e8d16c1e1e service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:18.811819 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-2c42c087-ed2a-4c61-b851-d96e14f4776e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f1c9901ebd541d18e1bb3f82ad237c0'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:18.824032 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 989/1967] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:14:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 01:14:18.840518 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-2c42c087-ed2a-4c61-b851-d96e14f4776e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f1c9901ebd541d18e1bb3f82ad237c0'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:18.850793 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-2c42c087-ed2a-4c61-b851-d96e14f4776e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f1c9901ebd541d18e1bb3f82ad237c0'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:18.852173 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-2c42c087-ed2a-4c61-b851-d96e14f4776e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f1c9901ebd541d18e1bb3f82ad237c0'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:18.864839 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-2c42c087-ed2a-4c61-b851-d96e14f4776e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f1c9901ebd541d18e1bb3f82ad237c0'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:18.875904 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 979/1968] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:14:18 2022] DELETE /identity/v3/users/2f1c9901ebd541d18e1bb3f82ad237c0 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:14:18.881170 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b94c9-ddc9-47cb-a9fb-d83e6f7ad52e None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:18.881620 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b94c9-ddc9-47cb-a9fb-d83e6f7ad52e None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:18.881716 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b94c9-ddc9-47cb-a9fb-d83e6f7ad52e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:18.914219 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-7c7b94c9-ddc9-47cb-a9fb-d83e6f7ad52e None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:18.919985 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-7c7b94c9-ddc9-47cb-a9fb-d83e6f7ad52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:19.025770 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 990/1969] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:19.038418 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c42c087-ed2a-4c61-b851-d96e14f4776e admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:19.039926 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-2c42c087-ed2a-4c61-b851-d96e14f4776e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:19.160686 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c42c087-ed2a-4c61-b851-d96e14f4776e admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:19.176565 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d66cf39-7986-4ac5-99a3-408c366ac424 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:19.178521 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d66cf39-7986-4ac5-99a3-408c366ac424 service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:19.180077 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d66cf39-7986-4ac5-99a3-408c366ac424 service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:19.180484 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d66cf39-7986-4ac5-99a3-408c366ac424 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:19.205785 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d66cf39-7986-4ac5-99a3-408c366ac424 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:19.207234 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d66cf39-7986-4ac5-99a3-408c366ac424 service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:19.226099 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 980/1970] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:14:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:14:19.648545 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b94c9-ddc9-47cb-a9fb-d83e6f7ad52e None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:19.652684 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-7c7b94c9-ddc9-47cb-a9fb-d83e6f7ad52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:19.705963 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b94c9-ddc9-47cb-a9fb-d83e6f7ad52e None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:19.718582 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd27fcd4-21fd-4d40-a5cb-9497ca6d5153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:19.719324 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27fcd4-21fd-4d40-a5cb-9497ca6d5153 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:19.719587 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27fcd4-21fd-4d40-a5cb-9497ca6d5153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:19.720005 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27fcd4-21fd-4d40-a5cb-9497ca6d5153 admin admin] PATH_INFO: `/v3/projects/d17cfb790998450895f07969087655e9` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:19.721215 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd27fcd4-21fd-4d40-a5cb-9497ca6d5153 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d17cfb790998450895f07969087655e9)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:19.723142 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd27fcd4-21fd-4d40-a5cb-9497ca6d5153 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:19.803091 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-cd27fcd4-21fd-4d40-a5cb-9497ca6d5153 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd17cfb790998450895f07969087655e9', 'request_id': 'req-cd27fcd4-21fd-4d40-a5cb-9497ca6d5153'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:19.809944 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-cd27fcd4-21fd-4d40-a5cb-9497ca6d5153 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd17cfb790998450895f07969087655e9', 'request_id': 'req-cd27fcd4-21fd-4d40-a5cb-9497ca6d5153'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:19.817774 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-cd27fcd4-21fd-4d40-a5cb-9497ca6d5153 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd17cfb790998450895f07969087655e9', 'request_id': 'req-cd27fcd4-21fd-4d40-a5cb-9497ca6d5153'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:19.832656 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-cd27fcd4-21fd-4d40-a5cb-9497ca6d5153 admin admin] The token cache is being invalidate because project d17cfb790998450895f07969087655e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:14:19.833117 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-cd27fcd4-21fd-4d40-a5cb-9497ca6d5153 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:19.833927 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-cd27fcd4-21fd-4d40-a5cb-9497ca6d5153 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:19.836796 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 991/1971] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:14:19 2022] DELETE /identity/v3/projects/d17cfb790998450895f07969087655e9 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:19.845356 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d66cf39-7986-4ac5-99a3-408c366ac424 service neutron] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:19.861950 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d66cf39-7986-4ac5-99a3-408c366ac424 service neutron] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:19.932880 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5f608f-772e-46a2-b15f-c9af95fee243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:19.933545 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f608f-772e-46a2-b15f-c9af95fee243 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:19.933808 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f608f-772e-46a2-b15f-c9af95fee243 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:19.934088 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f608f-772e-46a2-b15f-c9af95fee243 admin admin] PATH_INFO: `/v3/users/f117d4d42ea24fb5a0116338ab988390` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:19.935225 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5f608f-772e-46a2-b15f-c9af95fee243 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f117d4d42ea24fb5a0116338ab988390)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:19.937891 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5f608f-772e-46a2-b15f-c9af95fee243 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:19.947640 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd27fcd4-21fd-4d40-a5cb-9497ca6d5153 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:19.950033 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-cd27fcd4-21fd-4d40-a5cb-9497ca6d5153 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:19.974846 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd27fcd4-21fd-4d40-a5cb-9497ca6d5153 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:19.989168 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b77fe-452e-4750-b306-4b1fe65e0ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:19.989810 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b77fe-452e-4750-b306-4b1fe65e0ed5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:19.990069 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b77fe-452e-4750-b306-4b1fe65e0ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:19.990343 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b77fe-452e-4750-b306-4b1fe65e0ed5 admin admin] PATH_INFO: `/v3/projects/dd159c1e58cb42e8ac2f55eaebf2cfd7` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:19.991888 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56b77fe-452e-4750-b306-4b1fe65e0ed5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd159c1e58cb42e8ac2f55eaebf2cfd7)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:19.993874 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56b77fe-452e-4750-b306-4b1fe65e0ed5 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:20.009389 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-ee5f608f-772e-46a2-b15f-c9af95fee243 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f117d4d42ea24fb5a0116338ab988390'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:20.013714 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:20.023820 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-ee5f608f-772e-46a2-b15f-c9af95fee243 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f117d4d42ea24fb5a0116338ab988390'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:20.026280 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:20.034034 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-ee5f608f-772e-46a2-b15f-c9af95fee243 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f117d4d42ea24fb5a0116338ab988390'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:20.035041 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-ee5f608f-772e-46a2-b15f-c9af95fee243 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f117d4d42ea24fb5a0116338ab988390'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:20.042976 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-ee5f608f-772e-46a2-b15f-c9af95fee243 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f117d4d42ea24fb5a0116338ab988390'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:20.051356 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 981/1972] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:14:19 2022] DELETE /identity/v3/users/f117d4d42ea24fb5a0116338ab988390 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:20.058148 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-e56b77fe-452e-4750-b306-4b1fe65e0ed5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd159c1e58cb42e8ac2f55eaebf2cfd7', 'request_id': 'req-e56b77fe-452e-4750-b306-4b1fe65e0ed5'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:20.065400 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-e56b77fe-452e-4750-b306-4b1fe65e0ed5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd159c1e58cb42e8ac2f55eaebf2cfd7', 'request_id': 'req-e56b77fe-452e-4750-b306-4b1fe65e0ed5'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:20.072809 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-e56b77fe-452e-4750-b306-4b1fe65e0ed5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd159c1e58cb42e8ac2f55eaebf2cfd7', 'request_id': 'req-e56b77fe-452e-4750-b306-4b1fe65e0ed5'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:20.087029 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-e56b77fe-452e-4750-b306-4b1fe65e0ed5 admin admin] The token cache is being invalidate because project dd159c1e58cb42e8ac2f55eaebf2cfd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:14:20.087763 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-e56b77fe-452e-4750-b306-4b1fe65e0ed5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:20.088578 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-e56b77fe-452e-4750-b306-4b1fe65e0ed5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:20.090222 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 992/1973] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:14:19 2022] DELETE /identity/v3/projects/dd159c1e58cb42e8ac2f55eaebf2cfd7 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:20.095396 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5f608f-772e-46a2-b15f-c9af95fee243 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:20.097138 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-ee5f608f-772e-46a2-b15f-c9af95fee243 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:20.199713 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5f608f-772e-46a2-b15f-c9af95fee243 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:20.206789 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c23c1be-c1bb-40c3-a23e-7336194fd954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:20.207535 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c23c1be-c1bb-40c3-a23e-7336194fd954 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:20.207883 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c23c1be-c1bb-40c3-a23e-7336194fd954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:20.208251 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c23c1be-c1bb-40c3-a23e-7336194fd954 admin admin] PATH_INFO: `/v3/users/3d9b1dd73605493da6a6f93f3bff4fd9` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:20.209590 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c23c1be-c1bb-40c3-a23e-7336194fd954 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d9b1dd73605493da6a6f93f3bff4fd9)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:20.211787 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c23c1be-c1bb-40c3-a23e-7336194fd954 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:20.306645 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-3c23c1be-c1bb-40c3-a23e-7336194fd954 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d9b1dd73605493da6a6f93f3bff4fd9'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:20.327929 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-3c23c1be-c1bb-40c3-a23e-7336194fd954 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d9b1dd73605493da6a6f93f3bff4fd9'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:20.347552 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-3c23c1be-c1bb-40c3-a23e-7336194fd954 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d9b1dd73605493da6a6f93f3bff4fd9'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:20.349142 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-3c23c1be-c1bb-40c3-a23e-7336194fd954 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d9b1dd73605493da6a6f93f3bff4fd9'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:20.360750 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-3c23c1be-c1bb-40c3-a23e-7336194fd954 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d9b1dd73605493da6a6f93f3bff4fd9'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:20.368349 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 982/1974] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:14:20 2022] DELETE /identity/v3/users/3d9b1dd73605493da6a6f93f3bff4fd9 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:20.481414 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b77fe-452e-4750-b306-4b1fe65e0ed5 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:20.484019 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-e56b77fe-452e-4750-b306-4b1fe65e0ed5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:20.597901 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b77fe-452e-4750-b306-4b1fe65e0ed5 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:20.772725 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e47f08-87a8-44de-870d-62d0b5880f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:20.773389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e47f08-87a8-44de-870d-62d0b5880f4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:20.773667 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e47f08-87a8-44de-870d-62d0b5880f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:20.774152 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e47f08-87a8-44de-870d-62d0b5880f4d admin admin] PATH_INFO: `/v3/projects/a760b6850ecf45938c0ee87e2e5f30e7` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:20.777274 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e47f08-87a8-44de-870d-62d0b5880f4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a760b6850ecf45938c0ee87e2e5f30e7)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:20.786152 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e47f08-87a8-44de-870d-62d0b5880f4d admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:20.849155 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c23c1be-c1bb-40c3-a23e-7336194fd954 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:20.851164 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-3c23c1be-c1bb-40c3-a23e-7336194fd954 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:20.870612 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c23c1be-c1bb-40c3-a23e-7336194fd954 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:20.888653 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-15e47f08-87a8-44de-870d-62d0b5880f4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a760b6850ecf45938c0ee87e2e5f30e7', 'request_id': 'req-15e47f08-87a8-44de-870d-62d0b5880f4d'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:20.897599 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:20.938838 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-15e47f08-87a8-44de-870d-62d0b5880f4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a760b6850ecf45938c0ee87e2e5f30e7', 'request_id': 'req-15e47f08-87a8-44de-870d-62d0b5880f4d'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:20.951896 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-15e47f08-87a8-44de-870d-62d0b5880f4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a760b6850ecf45938c0ee87e2e5f30e7', 'request_id': 'req-15e47f08-87a8-44de-870d-62d0b5880f4d'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:20.984288 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-15e47f08-87a8-44de-870d-62d0b5880f4d admin admin] The token cache is being invalidate because project a760b6850ecf45938c0ee87e2e5f30e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:14:20.985360 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-15e47f08-87a8-44de-870d-62d0b5880f4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:20.987496 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-15e47f08-87a8-44de-870d-62d0b5880f4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:20.989613 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bedec74-8710-4a67-a2bc-5a54aa530662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:20.993336 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 993/1975] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:14:20 2022] DELETE /identity/v3/projects/a760b6850ecf45938c0ee87e2e5f30e7 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:20.993500 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bedec74-8710-4a67-a2bc-5a54aa530662 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:20.993500 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bedec74-8710-4a67-a2bc-5a54aa530662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:20.993500 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bedec74-8710-4a67-a2bc-5a54aa530662 admin admin] PATH_INFO: `/v3/projects/91be089a98664017a69d71446142a61c` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:20.993743 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e47f08-87a8-44de-870d-62d0b5880f4d admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:20.996975 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-15e47f08-87a8-44de-870d-62d0b5880f4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:20.997574 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bedec74-8710-4a67-a2bc-5a54aa530662 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91be089a98664017a69d71446142a61c)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:21.003716 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bedec74-8710-4a67-a2bc-5a54aa530662 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:21.012315 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:21.041318 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e47f08-87a8-44de-870d-62d0b5880f4d admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:21.052240 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5e160d-7e03-4f00-9243-b3709696ff5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:21.053245 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e160d-7e03-4f00-9243-b3709696ff5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:21.053526 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e160d-7e03-4f00-9243-b3709696ff5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:21.053823 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e160d-7e03-4f00-9243-b3709696ff5c admin admin] PATH_INFO: `/v3/users/81c978962290481b828282a516f089bc` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:21.055223 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5e160d-7e03-4f00-9243-b3709696ff5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=81c978962290481b828282a516f089bc)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:21.060170 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5e160d-7e03-4f00-9243-b3709696ff5c admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:21.103468 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-6bedec74-8710-4a67-a2bc-5a54aa530662 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91be089a98664017a69d71446142a61c', 'request_id': 'req-6bedec74-8710-4a67-a2bc-5a54aa530662'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:21.120861 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-6bedec74-8710-4a67-a2bc-5a54aa530662 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91be089a98664017a69d71446142a61c', 'request_id': 'req-6bedec74-8710-4a67-a2bc-5a54aa530662'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:21.135023 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-6bedec74-8710-4a67-a2bc-5a54aa530662 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91be089a98664017a69d71446142a61c', 'request_id': 'req-6bedec74-8710-4a67-a2bc-5a54aa530662'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:21.147442 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-6bedec74-8710-4a67-a2bc-5a54aa530662 admin admin] The token cache is being invalidate because project 91be089a98664017a69d71446142a61c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81661) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:14:21.147767 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-6bedec74-8710-4a67-a2bc-5a54aa530662 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:21.148984 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-6bedec74-8710-4a67-a2bc-5a54aa530662 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:21.156325 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 983/1976] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:14:20 2022] DELETE /identity/v3/projects/91be089a98664017a69d71446142a61c => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:21.157324 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bedec74-8710-4a67-a2bc-5a54aa530662 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:21.161541 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-9e5e160d-7e03-4f00-9243-b3709696ff5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81c978962290481b828282a516f089bc'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:21.164256 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-6bedec74-8710-4a67-a2bc-5a54aa530662 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:21.197941 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-9e5e160d-7e03-4f00-9243-b3709696ff5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81c978962290481b828282a516f089bc'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:21.212187 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-9e5e160d-7e03-4f00-9243-b3709696ff5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81c978962290481b828282a516f089bc'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:21.224985 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-9e5e160d-7e03-4f00-9243-b3709696ff5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81c978962290481b828282a516f089bc'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:21.225793 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-9e5e160d-7e03-4f00-9243-b3709696ff5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81c978962290481b828282a516f089bc'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:21.242779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 994/1977] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:14:20 2022] DELETE /identity/v3/users/81c978962290481b828282a516f089bc => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:21.244414 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ce615-b6e2-48b9-9e03-8327bbf90953 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:21.244803 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ce615-b6e2-48b9-9e03-8327bbf90953 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:21.245293 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ce615-b6e2-48b9-9e03-8327bbf90953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:21.270813 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bedec74-8710-4a67-a2bc-5a54aa530662 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:21.282754 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-b76ce615-b6e2-48b9-9e03-8327bbf90953 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:21.284439 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-b76ce615-b6e2-48b9-9e03-8327bbf90953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:21.356573 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eabeac7-015c-48a5-b415-d4b34a1a952b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:21.357466 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eabeac7-015c-48a5-b415-d4b34a1a952b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:21.357750 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eabeac7-015c-48a5-b415-d4b34a1a952b admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:21.358024 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eabeac7-015c-48a5-b415-d4b34a1a952b admin admin] PATH_INFO: `/v3/users/126eae716681430b8a08c798ef218b68` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:21.361055 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eabeac7-015c-48a5-b415-d4b34a1a952b admin admin] RBAC: Authorizing `identity:delete_user(user_id=126eae716681430b8a08c798ef218b68)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:21.363048 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eabeac7-015c-48a5-b415-d4b34a1a952b admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:21.368319 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 995/1978] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:21.368985 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929cabcb-4097-4165-b4cb-3eef1f247427 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:21.369266 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929cabcb-4097-4165-b4cb-3eef1f247427 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:21.369541 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929cabcb-4097-4165-b4cb-3eef1f247427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:21.395371 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-929cabcb-4097-4165-b4cb-3eef1f247427 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:21.397051 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-929cabcb-4097-4165-b4cb-3eef1f247427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:21.409629 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 996/1979] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:21.409629 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b597ebc7-15df-4986-893d-862e79b6ca35 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:21.409629 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b597ebc7-15df-4986-893d-862e79b6ca35 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:21.409629 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b597ebc7-15df-4986-893d-862e79b6ca35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:21.432001 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-b597ebc7-15df-4986-893d-862e79b6ca35 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:21.433252 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-b597ebc7-15df-4986-893d-862e79b6ca35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:21.434842 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:21.440148 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-7eabeac7-015c-48a5-b415-d4b34a1a952b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '126eae716681430b8a08c798ef218b68'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:21.442498 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 997/1980] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:21.446029 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b597ebc7-15df-4986-893d-862e79b6ca35 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:21.446029 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-b597ebc7-15df-4986-893d-862e79b6ca35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:21.458110 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-7eabeac7-015c-48a5-b415-d4b34a1a952b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '126eae716681430b8a08c798ef218b68'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:21.465792 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-7eabeac7-015c-48a5-b415-d4b34a1a952b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '126eae716681430b8a08c798ef218b68'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:21.466734 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-7eabeac7-015c-48a5-b415-d4b34a1a952b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '126eae716681430b8a08c798ef218b68'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:21.477737 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-7eabeac7-015c-48a5-b415-d4b34a1a952b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '126eae716681430b8a08c798ef218b68'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:21.487538 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 984/1981] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:14:21 2022] DELETE /identity/v3/users/126eae716681430b8a08c798ef218b68 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:21.488413 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eabeac7-015c-48a5-b415-d4b34a1a952b admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:21.491034 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-7eabeac7-015c-48a5-b415-d4b34a1a952b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:21.499067 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:21.516803 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b597ebc7-15df-4986-893d-862e79b6ca35 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:21.576385 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eabeac7-015c-48a5-b415-d4b34a1a952b admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:21.585990 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9893190-485e-4201-90f4-eba6883d2cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:21.587418 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9893190-485e-4201-90f4-eba6883d2cff admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:21.588294 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9893190-485e-4201-90f4-eba6883d2cff admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:21.589357 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9893190-485e-4201-90f4-eba6883d2cff admin admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:21.590355 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9893190-485e-4201-90f4-eba6883d2cff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:14:21.590891 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9893190-485e-4201-90f4-eba6883d2cff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:21.591136 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186c9036-be06-441d-9ccb-dac0902b17dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:21.592191 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c9036-be06-441d-9ccb-dac0902b17dc service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:21.592191 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c9036-be06-441d-9ccb-dac0902b17dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:21.592473 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c9036-be06-441d-9ccb-dac0902b17dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:21.593823 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9893190-485e-4201-90f4-eba6883d2cff admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:21.594549 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-186c9036-be06-441d-9ccb-dac0902b17dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:21.609818 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 985/1982] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:14:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:14:21.614838 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186c9036-be06-441d-9ccb-dac0902b17dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:21.615505 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9893190-485e-4201-90f4-eba6883d2cff admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:21.616659 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186c9036-be06-441d-9ccb-dac0902b17dc service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:21.630015 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 998/1983] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:14:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:14:21.631525 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9893190-485e-4201-90f4-eba6883d2cff admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:21.643143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5295b80e-0e7f-45f1-a470-e6e43e70c52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:21.643143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5295b80e-0e7f-45f1-a470-e6e43e70c52a admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:21.643143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5295b80e-0e7f-45f1-a470-e6e43e70c52a admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:21.643550 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5295b80e-0e7f-45f1-a470-e6e43e70c52a admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:21.644221 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5295b80e-0e7f-45f1-a470-e6e43e70c52a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:21.646240 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5295b80e-0e7f-45f1-a470-e6e43e70c52a admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:21.690906 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 986/1984] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:14:21 2022] POST /identity/v3/projects => generated 371 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:14:21.695826 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186c9036-be06-441d-9ccb-dac0902b17dc service neutron] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:21.718224 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186c9036-be06-441d-9ccb-dac0902b17dc service neutron] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:21.821726 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48d0c04-fac4-4c42-baff-baf4f9c2afe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:21.824294 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48d0c04-fac4-4c42-baff-baf4f9c2afe6 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:21.824294 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48d0c04-fac4-4c42-baff-baf4f9c2afe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:21.824294 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48d0c04-fac4-4c42-baff-baf4f9c2afe6 admin admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:21.824294 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48d0c04-fac4-4c42-baff-baf4f9c2afe6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:21.827894 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48d0c04-fac4-4c42-baff-baf4f9c2afe6 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:21.869757 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 999/1985] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:14:21 2022] POST /identity/v3/users => generated 286 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 01:14:21.876256 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5295b80e-0e7f-45f1-a470-e6e43e70c52a admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:21.894815 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5295b80e-0e7f-45f1-a470-e6e43e70c52a admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:21.926267 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7e0f46-dbcb-48be-8f88-69da06de03c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:21.926900 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e0f46-dbcb-48be-8f88-69da06de03c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:21.927185 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e0f46-dbcb-48be-8f88-69da06de03c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:21.927462 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e0f46-dbcb-48be-8f88-69da06de03c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:21.928120 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7e0f46-dbcb-48be-8f88-69da06de03c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:21.930152 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7e0f46-dbcb-48be-8f88-69da06de03c3 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:21.950041 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 987/1986] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:14:21 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:14:21.950956 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48d0c04-fac4-4c42-baff-baf4f9c2afe6 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:21.962840 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48d0c04-fac4-4c42-baff-baf4f9c2afe6 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:21.970226 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7e0f46-dbcb-48be-8f88-69da06de03c3 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:21.974080 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-cd7e0f46-dbcb-48be-8f88-69da06de03c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:21.985332 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cecae9-d60a-4da9-a03e-df53cc03ea97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:21.986140 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cecae9-d60a-4da9-a03e-df53cc03ea97 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:21.986425 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cecae9-d60a-4da9-a03e-df53cc03ea97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:21.986704 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cecae9-d60a-4da9-a03e-df53cc03ea97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:21.987662 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cecae9-d60a-4da9-a03e-df53cc03ea97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:21.990220 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cecae9-d60a-4da9-a03e-df53cc03ea97 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:22.002732 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7e0f46-dbcb-48be-8f88-69da06de03c3 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:22.011964 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1000/1987] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:14:21 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 01:14:22.012159 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02b1cd4-a986-48b3-97fa-a500fad62e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:22.012159 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b1cd4-a986-48b3-97fa-a500fad62e53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:22.012658 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b1cd4-a986-48b3-97fa-a500fad62e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:22.013050 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b1cd4-a986-48b3-97fa-a500fad62e53 admin admin] PATH_INFO: `/v3/projects/d986d13fb1c44eebb563871b2ef748e7` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:22.013726 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cecae9-d60a-4da9-a03e-df53cc03ea97 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:22.014858 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02b1cd4-a986-48b3-97fa-a500fad62e53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d986d13fb1c44eebb563871b2ef748e7)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:22.017982 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02b1cd4-a986-48b3-97fa-a500fad62e53 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:22.025436 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cecae9-d60a-4da9-a03e-df53cc03ea97 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:22.030319 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6da43d-9af3-48fc-91e1-53609a46e965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:22.030995 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6da43d-9af3-48fc-91e1-53609a46e965 admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:22.031429 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6da43d-9af3-48fc-91e1-53609a46e965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:22.031684 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6da43d-9af3-48fc-91e1-53609a46e965 admin admin] PATH_INFO: `/v3/projects/6b2a99457ced46a5a4411d35efbe736e/users/c06f498366d54dba8a92f6c66d3bae0e/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:22.048642 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6da43d-9af3-48fc-91e1-53609a46e965 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b2a99457ced46a5a4411d35efbe736e, user_id=c06f498366d54dba8a92f6c66d3bae0e, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:22.051647 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6da43d-9af3-48fc-91e1-53609a46e965 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:22.068659 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1001/1988] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:14:22 2022] PUT /identity/v3/projects/6b2a99457ced46a5a4411d35efbe736e/users/c06f498366d54dba8a92f6c66d3bae0e/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:22.082374 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c150fee7-3938-4d77-8846-13cb7a52b54d None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:22.082935 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c150fee7-3938-4d77-8846-13cb7a52b54d None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:22.083392 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c150fee7-3938-4d77-8846-13cb7a52b54d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:22.100367 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-b02b1cd4-a986-48b3-97fa-a500fad62e53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd986d13fb1c44eebb563871b2ef748e7', 'request_id': 'req-b02b1cd4-a986-48b3-97fa-a500fad62e53'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:22.111846 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-b02b1cd4-a986-48b3-97fa-a500fad62e53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd986d13fb1c44eebb563871b2ef748e7', 'request_id': 'req-b02b1cd4-a986-48b3-97fa-a500fad62e53'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:22.122226 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-b02b1cd4-a986-48b3-97fa-a500fad62e53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd986d13fb1c44eebb563871b2ef748e7', 'request_id': 'req-b02b1cd4-a986-48b3-97fa-a500fad62e53'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:22.123546 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-c150fee7-3938-4d77-8846-13cb7a52b54d None None] MFA Rules not processed for user `c06f498366d54dba8a92f6c66d3bae0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:22.125326 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-c150fee7-3938-4d77-8846-13cb7a52b54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:22.145375 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-b02b1cd4-a986-48b3-97fa-a500fad62e53 admin admin] The token cache is being invalidate because project d986d13fb1c44eebb563871b2ef748e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81661) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:14:22.146034 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-b02b1cd4-a986-48b3-97fa-a500fad62e53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:22.146925 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-b02b1cd4-a986-48b3-97fa-a500fad62e53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:22.149310 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 988/1989] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:14:21 2022] DELETE /identity/v3/projects/d986d13fb1c44eebb563871b2ef748e7 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:22.155513 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120bb915-58a2-4490-a1e2-aeab12374acd None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:22.156249 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120bb915-58a2-4490-a1e2-aeab12374acd None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:22.156649 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120bb915-58a2-4490-a1e2-aeab12374acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:22.171383 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:DELETE=11,keystone:SELECT=86,keystone:INSERT=3 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:22.189565 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-120bb915-58a2-4490-a1e2-aeab12374acd None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:22.191485 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-120bb915-58a2-4490-a1e2-aeab12374acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:22.211101 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:22.260902 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1002/1990] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:14:22.260902 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c150fee7-3938-4d77-8846-13cb7a52b54d None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:22.261651 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-c150fee7-3938-4d77-8846-13cb7a52b54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:22.281311 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 989/1991] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:22.281311 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120bb915-58a2-4490-a1e2-aeab12374acd None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:22.281311 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-120bb915-58a2-4490-a1e2-aeab12374acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:22.364471 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c150fee7-3938-4d77-8846-13cb7a52b54d None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:22.376183 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72830b5b-af68-4923-b34c-98fcc6aa84fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:22.376828 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72830b5b-af68-4923-b34c-98fcc6aa84fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:22.377107 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72830b5b-af68-4923-b34c-98fcc6aa84fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:22.377380 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72830b5b-af68-4923-b34c-98fcc6aa84fa admin admin] PATH_INFO: `/v3/projects/20824692a9a64ceeb7e81aad2470ff3b` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:22.380875 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72830b5b-af68-4923-b34c-98fcc6aa84fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=20824692a9a64ceeb7e81aad2470ff3b)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:22.381673 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120bb915-58a2-4490-a1e2-aeab12374acd None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:22.382891 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72830b5b-af68-4923-b34c-98fcc6aa84fa admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:22.392753 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78294172-9a53-4e29-b88f-270c2ff6fa2b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1acfa7a09d34e2a9045c3f3fbea0f52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:22.393829 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78294172-9a53-4e29-b88f-270c2ff6fa2b service glance] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:22.394218 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78294172-9a53-4e29-b88f-270c2ff6fa2b service glance] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:22.394568 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78294172-9a53-4e29-b88f-270c2ff6fa2b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:22.410055 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78294172-9a53-4e29-b88f-270c2ff6fa2b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:22.411515 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78294172-9a53-4e29-b88f-270c2ff6fa2b service glance] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:22.427281 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 990/1992] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:14:22 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:14:22.428445 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736463a4-21ce-4c87-bf06-770fe4349697 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:22.428730 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736463a4-21ce-4c87-bf06-770fe4349697 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:22.429398 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736463a4-21ce-4c87-bf06-770fe4349697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:22.462270 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-736463a4-21ce-4c87-bf06-770fe4349697 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:22.464216 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-736463a4-21ce-4c87-bf06-770fe4349697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:22.497340 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 991/1993] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:22.504823 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736463a4-21ce-4c87-bf06-770fe4349697 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:22.510245 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-72830b5b-af68-4923-b34c-98fcc6aa84fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20824692a9a64ceeb7e81aad2470ff3b', 'request_id': 'req-72830b5b-af68-4923-b34c-98fcc6aa84fa'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:22.526670 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736463a4-21ce-4c87-bf06-770fe4349697 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:22.544973 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-72830b5b-af68-4923-b34c-98fcc6aa84fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20824692a9a64ceeb7e81aad2470ff3b', 'request_id': 'req-72830b5b-af68-4923-b34c-98fcc6aa84fa'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:22.556653 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-72830b5b-af68-4923-b34c-98fcc6aa84fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20824692a9a64ceeb7e81aad2470ff3b', 'request_id': 'req-72830b5b-af68-4923-b34c-98fcc6aa84fa'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:22.573360 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-72830b5b-af68-4923-b34c-98fcc6aa84fa admin admin] The token cache is being invalidate because project 20824692a9a64ceeb7e81aad2470ff3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:14:22.573748 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-72830b5b-af68-4923-b34c-98fcc6aa84fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:22.574443 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-72830b5b-af68-4923-b34c-98fcc6aa84fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:22.576004 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1003/1994] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:14:22 2022] DELETE /identity/v3/projects/20824692a9a64ceeb7e81aad2470ff3b => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:22.581963 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72830b5b-af68-4923-b34c-98fcc6aa84fa admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:22.586784 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-72830b5b-af68-4923-b34c-98fcc6aa84fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:22.674910 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c51f8a-ae94-4161-9872-ed4cb890e08e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:22.676204 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c51f8a-ae94-4161-9872-ed4cb890e08e admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:22.676490 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c51f8a-ae94-4161-9872-ed4cb890e08e admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:22.676766 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c51f8a-ae94-4161-9872-ed4cb890e08e admin admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:22.677435 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c51f8a-ae94-4161-9872-ed4cb890e08e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:14:22.677745 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c51f8a-ae94-4161-9872-ed4cb890e08e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:22.682970 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c51f8a-ae94-4161-9872-ed4cb890e08e admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:22.723509 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 992/1995] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:14:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:14:22.727512 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c51f8a-ae94-4161-9872-ed4cb890e08e admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:22.730784 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-e5c51f8a-ae94-4161-9872-ed4cb890e08e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:22.756383 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c51f8a-ae94-4161-9872-ed4cb890e08e admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:22.760185 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72830b5b-af68-4923-b34c-98fcc6aa84fa admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:22.767442 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f04d609-e83c-49f3-a722-109fd06d8688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:22.768911 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04d609-e83c-49f3-a722-109fd06d8688 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:22.770246 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04d609-e83c-49f3-a722-109fd06d8688 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:22.770532 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04d609-e83c-49f3-a722-109fd06d8688 admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:22.771193 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f04d609-e83c-49f3-a722-109fd06d8688 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:22.773237 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f04d609-e83c-49f3-a722-109fd06d8688 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:22.775354 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5836e3-deb9-4172-b3bf-174dbcd87706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:22.776324 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5836e3-deb9-4172-b3bf-174dbcd87706 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:22.776813 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5836e3-deb9-4172-b3bf-174dbcd87706 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:22.777647 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5836e3-deb9-4172-b3bf-174dbcd87706 admin admin] PATH_INFO: `/v3/users/b16e336cbee349de97bbc0d8c6c73c2e` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:22.779030 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5836e3-deb9-4172-b3bf-174dbcd87706 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b16e336cbee349de97bbc0d8c6c73c2e)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:22.782179 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5836e3-deb9-4172-b3bf-174dbcd87706 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:22.804352 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:22.836058 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 993/1996] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:14:22 2022] POST /identity/v3/projects => generated 381 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:14:22.840501 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f04d609-e83c-49f3-a722-109fd06d8688 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:22.859811 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-3d5836e3-deb9-4172-b3bf-174dbcd87706 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b16e336cbee349de97bbc0d8c6c73c2e'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:22.868962 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:22.891718 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-3d5836e3-deb9-4172-b3bf-174dbcd87706 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b16e336cbee349de97bbc0d8c6c73c2e'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:22.915817 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-3d5836e3-deb9-4172-b3bf-174dbcd87706 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b16e336cbee349de97bbc0d8c6c73c2e'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:22.928814 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-3d5836e3-deb9-4172-b3bf-174dbcd87706 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b16e336cbee349de97bbc0d8c6c73c2e'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:22.930506 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-3d5836e3-deb9-4172-b3bf-174dbcd87706 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b16e336cbee349de97bbc0d8c6c73c2e'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:22.948397 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f04d609-e83c-49f3-a722-109fd06d8688 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:22.950039 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1004/1997] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:14:22 2022] DELETE /identity/v3/users/b16e336cbee349de97bbc0d8c6c73c2e => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:22.954109 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d2a9c-c6ca-4415-bd20-eee0be435786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:22.954810 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d2a9c-c6ca-4415-bd20-eee0be435786 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:22.955098 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d2a9c-c6ca-4415-bd20-eee0be435786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:22.955373 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d2a9c-c6ca-4415-bd20-eee0be435786 admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:22.956176 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363d2a9c-c6ca-4415-bd20-eee0be435786 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:22.958604 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363d2a9c-c6ca-4415-bd20-eee0be435786 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:22.983823 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 994/1998] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:14:22 2022] POST /identity/v3/users => generated 291 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:14:22.988768 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5836e3-deb9-4172-b3bf-174dbcd87706 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:22.990157 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-3d5836e3-deb9-4172-b3bf-174dbcd87706 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:23.103079 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5836e3-deb9-4172-b3bf-174dbcd87706 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:23.109772 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3dd5d6-0f3c-4a4c-a84b-35b58ec0c440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:23.110558 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3dd5d6-0f3c-4a4c-a84b-35b58ec0c440 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:23.111066 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3dd5d6-0f3c-4a4c-a84b-35b58ec0c440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:23.111209 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3dd5d6-0f3c-4a4c-a84b-35b58ec0c440 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:23.111989 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3dd5d6-0f3c-4a4c-a84b-35b58ec0c440 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:23.113751 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3dd5d6-0f3c-4a4c-a84b-35b58ec0c440 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:23.125990 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1005/1999] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:14:23.130415 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d2a9c-c6ca-4415-bd20-eee0be435786 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:23.147782 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d2a9c-c6ca-4415-bd20-eee0be435786 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:23.172056 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb72d159-464c-4550-9664-1d23c0316f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:23.172878 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72d159-464c-4550-9664-1d23c0316f66 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:23.173236 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72d159-464c-4550-9664-1d23c0316f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:23.173569 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72d159-464c-4550-9664-1d23c0316f66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:23.174293 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb72d159-464c-4550-9664-1d23c0316f66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:23.176025 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb72d159-464c-4550-9664-1d23c0316f66 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:23.199185 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 995/2000] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:14:23 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:14:23.201057 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3dd5d6-0f3c-4a4c-a84b-35b58ec0c440 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:23.213353 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3dd5d6-0f3c-4a4c-a84b-35b58ec0c440 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:23.218951 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d738bc6-e90c-43fb-8f10-58b6900d8b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:23.219778 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d738bc6-e90c-43fb-8f10-58b6900d8b8d admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:23.220171 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d738bc6-e90c-43fb-8f10-58b6900d8b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:23.220637 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d738bc6-e90c-43fb-8f10-58b6900d8b8d admin admin] PATH_INFO: `/v3/projects/b66b5341c5424e258079815a66b79d52/users/845100e2f2184568830fe72f7442c185/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:23.250128 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d738bc6-e90c-43fb-8f10-58b6900d8b8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b66b5341c5424e258079815a66b79d52, user_id=845100e2f2184568830fe72f7442c185, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:23.253650 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d738bc6-e90c-43fb-8f10-58b6900d8b8d admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:23.274443 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1006/2001] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:14:23 2022] PUT /identity/v3/projects/b66b5341c5424e258079815a66b79d52/users/845100e2f2184568830fe72f7442c185/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:14:23.286052 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9602be4-2af6-46f9-95cb-804a4c82788e None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:23.287149 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9602be4-2af6-46f9-95cb-804a4c82788e None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:23.287149 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9602be4-2af6-46f9-95cb-804a4c82788e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:23.319180 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-a9602be4-2af6-46f9-95cb-804a4c82788e None None] MFA Rules not processed for user `845100e2f2184568830fe72f7442c185`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:23.320490 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-a9602be4-2af6-46f9-95cb-804a4c82788e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:23.402383 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 996/2002] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:23 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:23.407758 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d738bc6-e90c-43fb-8f10-58b6900d8b8d admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:23.489202 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d738bc6-e90c-43fb-8f10-58b6900d8b8d admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:23.495082 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2668f20-03da-48e9-8d6a-68419b7d3ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:23.495756 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2668f20-03da-48e9-8d6a-68419b7d3ec3 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:23.496040 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2668f20-03da-48e9-8d6a-68419b7d3ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:23.496393 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2668f20-03da-48e9-8d6a-68419b7d3ec3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:23.497052 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2668f20-03da-48e9-8d6a-68419b7d3ec3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:23.499404 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2668f20-03da-48e9-8d6a-68419b7d3ec3 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:23.510673 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:23.538250 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1007/2003] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:14:23 2022] POST /identity/v3/projects => generated 381 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:14:23.542922 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9602be4-2af6-46f9-95cb-804a4c82788e None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:23.560792 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9602be4-2af6-46f9-95cb-804a4c82788e None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:23.574520 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2668f20-03da-48e9-8d6a-68419b7d3ec3 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:23.577780 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-f2668f20-03da-48e9-8d6a-68419b7d3ec3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:23.679471 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2668f20-03da-48e9-8d6a-68419b7d3ec3 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:23.684893 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800afe24-d23f-4d74-be30-83eec6f6403a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:23.685685 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800afe24-d23f-4d74-be30-83eec6f6403a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:23.686066 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800afe24-d23f-4d74-be30-83eec6f6403a admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:23.687071 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800afe24-d23f-4d74-be30-83eec6f6403a admin admin] PATH_INFO: `/v3/projects/a31d05c114444ca7948f87e2a5d591a4` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:23.688323 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800afe24-d23f-4d74-be30-83eec6f6403a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a31d05c114444ca7948f87e2a5d591a4)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:23.688864 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcac9e3-a8ac-4013-a78a-862d3c5873c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:23.689504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcac9e3-a8ac-4013-a78a-862d3c5873c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:23.689918 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcac9e3-a8ac-4013-a78a-862d3c5873c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:23.690208 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcac9e3-a8ac-4013-a78a-862d3c5873c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:23.690310 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800afe24-d23f-4d74-be30-83eec6f6403a admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:23.691468 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcac9e3-a8ac-4013-a78a-862d3c5873c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:23.694165 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcac9e3-a8ac-4013-a78a-862d3c5873c0 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:23.727682 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 997/2004] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:14:23 2022] POST /identity/v3/users => generated 297 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:14:23.728822 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcac9e3-a8ac-4013-a78a-862d3c5873c0 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:23.740458 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcac9e3-a8ac-4013-a78a-862d3c5873c0 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:23.747545 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b916e9-4227-4a3b-a798-4eb826734b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:23.748471 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b916e9-4227-4a3b-a798-4eb826734b6c admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:23.749356 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b916e9-4227-4a3b-a798-4eb826734b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:23.749706 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b916e9-4227-4a3b-a798-4eb826734b6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:23.750633 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b916e9-4227-4a3b-a798-4eb826734b6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:23.753139 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b916e9-4227-4a3b-a798-4eb826734b6c admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:23.767611 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:23.782493 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-800afe24-d23f-4d74-be30-83eec6f6403a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a31d05c114444ca7948f87e2a5d591a4', 'request_id': 'req-800afe24-d23f-4d74-be30-83eec6f6403a'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:23.782801 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 998/2005] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:14:23 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:14:23.783593 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b916e9-4227-4a3b-a798-4eb826734b6c admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:23.795938 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-800afe24-d23f-4d74-be30-83eec6f6403a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a31d05c114444ca7948f87e2a5d591a4', 'request_id': 'req-800afe24-d23f-4d74-be30-83eec6f6403a'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:23.806398 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b916e9-4227-4a3b-a798-4eb826734b6c admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:23.810077 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-800afe24-d23f-4d74-be30-83eec6f6403a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a31d05c114444ca7948f87e2a5d591a4', 'request_id': 'req-800afe24-d23f-4d74-be30-83eec6f6403a'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:23.836172 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-800afe24-d23f-4d74-be30-83eec6f6403a admin admin] The token cache is being invalidate because project a31d05c114444ca7948f87e2a5d591a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:14:23.836562 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-800afe24-d23f-4d74-be30-83eec6f6403a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:23.837954 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-800afe24-d23f-4d74-be30-83eec6f6403a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:23.847737 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1008/2006] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:14:23 2022] DELETE /identity/v3/projects/a31d05c114444ca7948f87e2a5d591a4 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:23.849673 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd50943e-b729-4ff5-8472-268cc5c4fcd8 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:23.849965 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd50943e-b729-4ff5-8472-268cc5c4fcd8 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:23.850241 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd50943e-b729-4ff5-8472-268cc5c4fcd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:23.898870 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-dd50943e-b729-4ff5-8472-268cc5c4fcd8 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:23.900471 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-dd50943e-b729-4ff5-8472-268cc5c4fcd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:23.943987 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306565a7-e97c-4f4b-bf96-cc3df8fdb832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:23.945505 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306565a7-e97c-4f4b-bf96-cc3df8fdb832 admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:23.945977 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306565a7-e97c-4f4b-bf96-cc3df8fdb832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:23.946435 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306565a7-e97c-4f4b-bf96-cc3df8fdb832 admin admin] PATH_INFO: `/v3/projects/4297472c3efd4f67a08ed29c3e81d8d9/users/7286139a175841f9a6506f41c64d62dd/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:23.985466 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306565a7-e97c-4f4b-bf96-cc3df8fdb832 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4297472c3efd4f67a08ed29c3e81d8d9, user_id=7286139a175841f9a6506f41c64d62dd, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:23.990269 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306565a7-e97c-4f4b-bf96-cc3df8fdb832 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:24.018176 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 999/2007] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:14:23 2022] PUT /identity/v3/projects/4297472c3efd4f67a08ed29c3e81d8d9/users/7286139a175841f9a6506f41c64d62dd/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:24.026055 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1009/2008] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:24.026603 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306565a7-e97c-4f4b-bf96-cc3df8fdb832 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:24.038649 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f59e3-d45a-4129-8a02-3e3a98f716b0 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:24.038953 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f59e3-d45a-4129-8a02-3e3a98f716b0 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:24.039233 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f59e3-d45a-4129-8a02-3e3a98f716b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:24.042340 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-306565a7-e97c-4f4b-bf96-cc3df8fdb832 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:24.092151 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-c19f59e3-d45a-4129-8a02-3e3a98f716b0 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:24.098679 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-c19f59e3-d45a-4129-8a02-3e3a98f716b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:24.155199 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:24.224715 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306565a7-e97c-4f4b-bf96-cc3df8fdb832 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:24.242005 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5b4b79-da9e-4390-8bdf-865fba2753bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:24.243615 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b4b79-da9e-4390-8bdf-865fba2753bb admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:24.247357 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b4b79-da9e-4390-8bdf-865fba2753bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:24.247357 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b4b79-da9e-4390-8bdf-865fba2753bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:24.247357 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5b4b79-da9e-4390-8bdf-865fba2753bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:24.247357 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5b4b79-da9e-4390-8bdf-865fba2753bb admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:24.272297 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1000/2009] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:14:24.273970 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5b4b79-da9e-4390-8bdf-865fba2753bb admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:24.289586 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5b4b79-da9e-4390-8bdf-865fba2753bb admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:24.297714 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4b1c20-309f-4731-99ea-23966685e652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:24.300222 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b1c20-309f-4731-99ea-23966685e652 admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:24.300511 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b1c20-309f-4731-99ea-23966685e652 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:24.300790 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b1c20-309f-4731-99ea-23966685e652 admin admin] PATH_INFO: `/v3/projects/4297472c3efd4f67a08ed29c3e81d8d9/users/7286139a175841f9a6506f41c64d62dd/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:24.303934 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4b1c20-309f-4731-99ea-23966685e652 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4297472c3efd4f67a08ed29c3e81d8d9, user_id=7286139a175841f9a6506f41c64d62dd, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:24.308189 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4b1c20-309f-4731-99ea-23966685e652 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:24.309814 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1010/2010] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:24.313874 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19f59e3-d45a-4129-8a02-3e3a98f716b0 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:24.338166 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19f59e3-d45a-4129-8a02-3e3a98f716b0 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:24.348181 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1001/2011] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:14:24 2022] PUT /identity/v3/projects/4297472c3efd4f67a08ed29c3e81d8d9/users/7286139a175841f9a6506f41c64d62dd/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:14:24.356881 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48afaf7-f65b-4bcd-bf31-1db6a0edbc62 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:24.357615 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48afaf7-f65b-4bcd-bf31-1db6a0edbc62 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:24.358118 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48afaf7-f65b-4bcd-bf31-1db6a0edbc62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:24.410384 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-b48afaf7-f65b-4bcd-bf31-1db6a0edbc62 None None] MFA Rules not processed for user `7286139a175841f9a6506f41c64d62dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:24.415025 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-b48afaf7-f65b-4bcd-bf31-1db6a0edbc62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:24.450630 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e683b1-aec3-4467-8231-59a2a592b336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:24.452724 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e683b1-aec3-4467-8231-59a2a592b336 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:24.453247 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e683b1-aec3-4467-8231-59a2a592b336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:24.453755 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e683b1-aec3-4467-8231-59a2a592b336 admin admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:24.454888 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e683b1-aec3-4467-8231-59a2a592b336 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:14:24.455921 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e683b1-aec3-4467-8231-59a2a592b336 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:24.459316 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e683b1-aec3-4467-8231-59a2a592b336 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:24.480960 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1011/2012] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:14:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:14:24.485383 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e683b1-aec3-4467-8231-59a2a592b336 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:24.508495 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e683b1-aec3-4467-8231-59a2a592b336 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:24.524644 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c29755-7e23-4c0e-9955-ceb71552c525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:24.525276 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c29755-7e23-4c0e-9955-ceb71552c525 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:24.525553 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c29755-7e23-4c0e-9955-ceb71552c525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:24.525822 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c29755-7e23-4c0e-9955-ceb71552c525 admin admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:24.526496 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c29755-7e23-4c0e-9955-ceb71552c525 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:24.529790 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c29755-7e23-4c0e-9955-ceb71552c525 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:24.534277 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:24.557185 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1002/2013] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:24 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:24.569916 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48afaf7-f65b-4bcd-bf31-1db6a0edbc62 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:24.571439 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-b48afaf7-f65b-4bcd-bf31-1db6a0edbc62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:24.572270 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1012/2014] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:14:24 2022] POST /identity/v3/projects => generated 377 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:14:24.576316 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c29755-7e23-4c0e-9955-ceb71552c525 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:24.649597 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48afaf7-f65b-4bcd-bf31-1db6a0edbc62 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:24.690873 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c29755-7e23-4c0e-9955-ceb71552c525 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:24.698224 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91569bdb-cf4b-4332-a527-f2866e920b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:24.699252 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91569bdb-cf4b-4332-a527-f2866e920b92 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:24.699534 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91569bdb-cf4b-4332-a527-f2866e920b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:24.704184 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91569bdb-cf4b-4332-a527-f2866e920b92 admin admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:24.704562 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91569bdb-cf4b-4332-a527-f2866e920b92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:24.706775 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91569bdb-cf4b-4332-a527-f2866e920b92 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:24.744200 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3abe45-ab3d-4c6a-843a-83aff1d79f46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:24.744848 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3abe45-ab3d-4c6a-843a-83aff1d79f46 service cinder] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:24.747175 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3abe45-ab3d-4c6a-843a-83aff1d79f46 service cinder] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:24.748174 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3abe45-ab3d-4c6a-843a-83aff1d79f46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:24.750696 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1013/2015] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:14:24 2022] POST /identity/v3/users => generated 289 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 01:14:24.755772 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91569bdb-cf4b-4332-a527-f2866e920b92 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:24.779051 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91569bdb-cf4b-4332-a527-f2866e920b92 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:24.785696 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba210f1-300b-402d-aff5-166519095309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:24.786316 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba210f1-300b-402d-aff5-166519095309 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:24.786596 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba210f1-300b-402d-aff5-166519095309 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:24.786869 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba210f1-300b-402d-aff5-166519095309 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:24.791518 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba210f1-300b-402d-aff5-166519095309 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:24.793407 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba210f1-300b-402d-aff5-166519095309 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:24.815330 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1014/2016] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:14:24.819260 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba210f1-300b-402d-aff5-166519095309 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:24.829578 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba210f1-300b-402d-aff5-166519095309 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:24.834748 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af52a9a8-915c-4967-9886-0e518e46d9c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:24.835479 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af52a9a8-915c-4967-9886-0e518e46d9c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:24.835983 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af52a9a8-915c-4967-9886-0e518e46d9c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:24.836506 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af52a9a8-915c-4967-9886-0e518e46d9c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:24.838338 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af52a9a8-915c-4967-9886-0e518e46d9c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:24.841224 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af52a9a8-915c-4967-9886-0e518e46d9c1 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:24.847468 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3abe45-ab3d-4c6a-843a-83aff1d79f46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:24.848983 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3abe45-ab3d-4c6a-843a-83aff1d79f46 service cinder] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:24.849579 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:24.860497 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1015/2017] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 01:14:24.863720 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af52a9a8-915c-4967-9886-0e518e46d9c1 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:24.868239 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1003/2018] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:14:24 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 01:14:24.896704 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af52a9a8-915c-4967-9886-0e518e46d9c1 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:24.902317 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd53c642-e287-41a3-b322-b3bc5d0d8f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:24.903142 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd53c642-e287-41a3-b322-b3bc5d0d8f63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:24.903442 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd53c642-e287-41a3-b322-b3bc5d0d8f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:24.903690 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd53c642-e287-41a3-b322-b3bc5d0d8f63 admin admin] PATH_INFO: `/v3/projects/1dbe690577614c0e880bc13a3d659eb0/users/3f8fd6b58b184bac824348b6a19dec4f/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:24.949186 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd53c642-e287-41a3-b322-b3bc5d0d8f63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dbe690577614c0e880bc13a3d659eb0, user_id=3f8fd6b58b184bac824348b6a19dec4f, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:24.951787 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd53c642-e287-41a3-b322-b3bc5d0d8f63 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:24.969372 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1016/2019] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:14:24 2022] PUT /identity/v3/projects/1dbe690577614c0e880bc13a3d659eb0/users/3f8fd6b58b184bac824348b6a19dec4f/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:24.982895 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6739c73-7e35-42a9-b2eb-9509cb1d77af None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:24.984951 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6739c73-7e35-42a9-b2eb-9509cb1d77af None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:24.985414 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6739c73-7e35-42a9-b2eb-9509cb1d77af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:25.030641 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-c6739c73-7e35-42a9-b2eb-9509cb1d77af None None] MFA Rules not processed for user `3f8fd6b58b184bac824348b6a19dec4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:25.032114 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-c6739c73-7e35-42a9-b2eb-9509cb1d77af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:25.096564 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:25.143137 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1004/2020] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:25.144702 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd53c642-e287-41a3-b322-b3bc5d0d8f63 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:25.217177 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd53c642-e287-41a3-b322-b3bc5d0d8f63 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:25.222973 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d25909-b6ac-45d2-afe9-69a458fe4a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:25.223616 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d25909-b6ac-45d2-afe9-69a458fe4a33 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:25.223893 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d25909-b6ac-45d2-afe9-69a458fe4a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:25.224192 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d25909-b6ac-45d2-afe9-69a458fe4a33 admin admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:25.224836 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d25909-b6ac-45d2-afe9-69a458fe4a33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:25.226753 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d25909-b6ac-45d2-afe9-69a458fe4a33 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:25.267023 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1017/2021] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:14:25 2022] POST /identity/v3/projects => generated 377 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:14:25.268587 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6739c73-7e35-42a9-b2eb-9509cb1d77af None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:25.279958 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6739c73-7e35-42a9-b2eb-9509cb1d77af None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:25.362436 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25d7b5-070b-4fff-8edc-fcd53498cccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:25.364000 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25d7b5-070b-4fff-8edc-fcd53498cccc admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:25.364449 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25d7b5-070b-4fff-8edc-fcd53498cccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:25.364873 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25d7b5-070b-4fff-8edc-fcd53498cccc admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:25.365900 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb25d7b5-070b-4fff-8edc-fcd53498cccc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:25.369901 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb25d7b5-070b-4fff-8edc-fcd53498cccc admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:25.402403 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1005/2022] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:14:25 2022] POST /identity/v3/users => generated 295 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:14:25.406741 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d25909-b6ac-45d2-afe9-69a458fe4a33 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:25.420755 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d25909-b6ac-45d2-afe9-69a458fe4a33 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:25.423388 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25d7b5-070b-4fff-8edc-fcd53498cccc admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:25.427527 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71821b3d-2fdd-4fc2-86e0-8abd0f77a7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:25.428776 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71821b3d-2fdd-4fc2-86e0-8abd0f77a7f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:25.429284 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71821b3d-2fdd-4fc2-86e0-8abd0f77a7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:25.429793 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71821b3d-2fdd-4fc2-86e0-8abd0f77a7f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:25.432062 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71821b3d-2fdd-4fc2-86e0-8abd0f77a7f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:25.434175 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25d7b5-070b-4fff-8edc-fcd53498cccc admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:25.435388 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71821b3d-2fdd-4fc2-86e0-8abd0f77a7f0 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:25.444110 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf1c63c-e5f2-4355-9b6a-da38997423cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:25.444810 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf1c63c-e5f2-4355-9b6a-da38997423cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:25.445090 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf1c63c-e5f2-4355-9b6a-da38997423cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:25.445364 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf1c63c-e5f2-4355-9b6a-da38997423cb admin admin] PATH_INFO: `/v3/users/845100e2f2184568830fe72f7442c185` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:25.446610 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf1c63c-e5f2-4355-9b6a-da38997423cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=845100e2f2184568830fe72f7442c185)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:25.448678 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf1c63c-e5f2-4355-9b6a-da38997423cb admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:25.451121 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1018/2023] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:14:25.455765 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71821b3d-2fdd-4fc2-86e0-8abd0f77a7f0 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:25.466996 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71821b3d-2fdd-4fc2-86e0-8abd0f77a7f0 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:25.475689 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbd7eae-83c8-4477-bce4-bb70375e0ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:25.476314 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbd7eae-83c8-4477-bce4-bb70375e0ece admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:25.476596 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbd7eae-83c8-4477-bce4-bb70375e0ece admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:25.476982 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbd7eae-83c8-4477-bce4-bb70375e0ece admin admin] PATH_INFO: `/v3/projects/6a74f378827c4b52916d8183c76382cc/users/bfc2ce8eda0a4b678be76603be19cfa3/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:25.500201 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbd7eae-83c8-4477-bce4-bb70375e0ece admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a74f378827c4b52916d8183c76382cc, user_id=bfc2ce8eda0a4b678be76603be19cfa3, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:25.502467 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbd7eae-83c8-4477-bce4-bb70375e0ece admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:25.510863 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-7bf1c63c-e5f2-4355-9b6a-da38997423cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '845100e2f2184568830fe72f7442c185'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:25.522155 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1019/2024] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:14:25 2022] PUT /identity/v3/projects/6a74f378827c4b52916d8183c76382cc/users/bfc2ce8eda0a4b678be76603be19cfa3/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:14:25.525705 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbd7eae-83c8-4477-bce4-bb70375e0ece admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:25.530328 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-7bf1c63c-e5f2-4355-9b6a-da38997423cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '845100e2f2184568830fe72f7442c185'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:25.540992 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-7bf1c63c-e5f2-4355-9b6a-da38997423cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '845100e2f2184568830fe72f7442c185'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:25.541809 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-7bf1c63c-e5f2-4355-9b6a-da38997423cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '845100e2f2184568830fe72f7442c185'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:25.554440 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-7bf1c63c-e5f2-4355-9b6a-da38997423cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '845100e2f2184568830fe72f7442c185'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:25.569452 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1006/2025] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:14:25 2022] DELETE /identity/v3/users/845100e2f2184568830fe72f7442c185 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:25.574117 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93439f9-eb40-4011-b7c5-902523d7203e None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:25.574614 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93439f9-eb40-4011-b7c5-902523d7203e None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:25.575246 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93439f9-eb40-4011-b7c5-902523d7203e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:25.620306 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-d93439f9-eb40-4011-b7c5-902523d7203e None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:25.620486 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbd7eae-83c8-4477-bce4-bb70375e0ece admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:25.620600 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-d93439f9-eb40-4011-b7c5-902523d7203e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:25.693979 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:25.720439 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:25.756350 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e8430f-7267-40a6-809c-dd0b2e172dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:25.757044 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8430f-7267-40a6-809c-dd0b2e172dff admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:25.757417 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8430f-7267-40a6-809c-dd0b2e172dff admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:25.757692 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8430f-7267-40a6-809c-dd0b2e172dff admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:25.758721 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e8430f-7267-40a6-809c-dd0b2e172dff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:25.760702 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e8430f-7267-40a6-809c-dd0b2e172dff admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:25.766122 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1007/2026] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:25.778098 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93439f9-eb40-4011-b7c5-902523d7203e None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:25.779810 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-d93439f9-eb40-4011-b7c5-902523d7203e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:25.806917 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1020/2027] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 01:14:25.807921 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e8430f-7267-40a6-809c-dd0b2e172dff admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:25.814443 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-39e8430f-7267-40a6-809c-dd0b2e172dff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:25.834327 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e8430f-7267-40a6-809c-dd0b2e172dff admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:25.844071 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcab7bbb-89d9-420c-8581-ae6418a7a2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:25.852545 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab7bbb-89d9-420c-8581-ae6418a7a2ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:25.852856 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab7bbb-89d9-420c-8581-ae6418a7a2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:25.853145 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab7bbb-89d9-420c-8581-ae6418a7a2ac admin admin] PATH_INFO: `/v3/projects/6a74f378827c4b52916d8183c76382cc/users/bfc2ce8eda0a4b678be76603be19cfa3/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:25.855866 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcab7bbb-89d9-420c-8581-ae6418a7a2ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a74f378827c4b52916d8183c76382cc, user_id=bfc2ce8eda0a4b678be76603be19cfa3, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:25.862889 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcab7bbb-89d9-420c-8581-ae6418a7a2ac admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:25.882672 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93439f9-eb40-4011-b7c5-902523d7203e None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:25.886102 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1021/2028] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:14:25 2022] PUT /identity/v3/projects/6a74f378827c4b52916d8183c76382cc/users/bfc2ce8eda0a4b678be76603be19cfa3/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:25.897685 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b53b4ad-227f-438f-bcf8-20aea967033c None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:25.897997 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b53b4ad-227f-438f-bcf8-20aea967033c None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:25.898275 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b53b4ad-227f-438f-bcf8-20aea967033c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:25.949106 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-0b53b4ad-227f-438f-bcf8-20aea967033c None None] MFA Rules not processed for user `bfc2ce8eda0a4b678be76603be19cfa3`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:25.951377 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-0b53b4ad-227f-438f-bcf8-20aea967033c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:26.005339 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be33f12d-9b29-4b5c-a68a-4a4abc2552f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:26.006575 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be33f12d-9b29-4b5c-a68a-4a4abc2552f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:26.006901 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be33f12d-9b29-4b5c-a68a-4a4abc2552f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:26.007205 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be33f12d-9b29-4b5c-a68a-4a4abc2552f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:26.088909 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1022/2029] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:25 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:14:26.103689 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b53b4ad-227f-438f-bcf8-20aea967033c None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:26.105144 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-0b53b4ad-227f-438f-bcf8-20aea967033c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:26.125279 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be33f12d-9b29-4b5c-a68a-4a4abc2552f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:26.127375 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be33f12d-9b29-4b5c-a68a-4a4abc2552f0 service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:26.141523 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1008/2030] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:14:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:14:26.184720 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b53b4ad-227f-438f-bcf8-20aea967033c None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:26.190578 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649003b8-03fa-4ef4-aeea-634fa5d1de1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:26.191310 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649003b8-03fa-4ef4-aeea-634fa5d1de1b service cinder] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:26.191587 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649003b8-03fa-4ef4-aeea-634fa5d1de1b service cinder] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:26.191861 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649003b8-03fa-4ef4-aeea-634fa5d1de1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:26.194668 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-649003b8-03fa-4ef4-aeea-634fa5d1de1b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:26.299849 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:26.302174 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649003b8-03fa-4ef4-aeea-634fa5d1de1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:26.304330 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649003b8-03fa-4ef4-aeea-634fa5d1de1b service cinder] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:26.319488 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1023/2031] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:14:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 01:14:26.758835 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be33f12d-9b29-4b5c-a68a-4a4abc2552f0 service neutron] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:26.761456 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-be33f12d-9b29-4b5c-a68a-4a4abc2552f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:26.775679 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be33f12d-9b29-4b5c-a68a-4a4abc2552f0 service neutron] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:26.787773 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cba49f-3ba6-4bfa-97c3-570d4b70d23e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:26.788750 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cba49f-3ba6-4bfa-97c3-570d4b70d23e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:26.789060 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cba49f-3ba6-4bfa-97c3-570d4b70d23e admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:26.789407 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cba49f-3ba6-4bfa-97c3-570d4b70d23e admin admin] PATH_INFO: `/v3/projects/b66b5341c5424e258079815a66b79d52` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:26.790873 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cba49f-3ba6-4bfa-97c3-570d4b70d23e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b66b5341c5424e258079815a66b79d52)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:26.794540 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cba49f-3ba6-4bfa-97c3-570d4b70d23e admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:26.850169 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-c3cba49f-3ba6-4bfa-97c3-570d4b70d23e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b66b5341c5424e258079815a66b79d52', 'request_id': 'req-c3cba49f-3ba6-4bfa-97c3-570d4b70d23e'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:26.851934 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:26.868469 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-c3cba49f-3ba6-4bfa-97c3-570d4b70d23e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b66b5341c5424e258079815a66b79d52', 'request_id': 'req-c3cba49f-3ba6-4bfa-97c3-570d4b70d23e'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:26.876696 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-c3cba49f-3ba6-4bfa-97c3-570d4b70d23e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b66b5341c5424e258079815a66b79d52', 'request_id': 'req-c3cba49f-3ba6-4bfa-97c3-570d4b70d23e'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:26.888785 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-c3cba49f-3ba6-4bfa-97c3-570d4b70d23e admin admin] The token cache is being invalidate because project b66b5341c5424e258079815a66b79d52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81661) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:14:26.889163 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-c3cba49f-3ba6-4bfa-97c3-570d4b70d23e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:26.890065 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-c3cba49f-3ba6-4bfa-97c3-570d4b70d23e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:26.892302 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1009/2032] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:14:26 2022] DELETE /identity/v3/projects/b66b5341c5424e258079815a66b79d52 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:26.896787 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649003b8-03fa-4ef4-aeea-634fa5d1de1b service cinder] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:26.912051 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649003b8-03fa-4ef4-aeea-634fa5d1de1b service cinder] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:26.983591 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0062cbd-3ba7-4dbe-8cfc-b1fe65ee2edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:26.984312 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0062cbd-3ba7-4dbe-8cfc-b1fe65ee2edf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:26.984598 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0062cbd-3ba7-4dbe-8cfc-b1fe65ee2edf admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:26.984883 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0062cbd-3ba7-4dbe-8cfc-b1fe65ee2edf admin admin] PATH_INFO: `/v3/users/7286139a175841f9a6506f41c64d62dd` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:26.986122 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0062cbd-3ba7-4dbe-8cfc-b1fe65ee2edf admin admin] RBAC: Authorizing `identity:delete_user(user_id=7286139a175841f9a6506f41c64d62dd)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:26.988185 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0062cbd-3ba7-4dbe-8cfc-b1fe65ee2edf admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:27.055323 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-c0062cbd-3ba7-4dbe-8cfc-b1fe65ee2edf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7286139a175841f9a6506f41c64d62dd'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:27.095136 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-c0062cbd-3ba7-4dbe-8cfc-b1fe65ee2edf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7286139a175841f9a6506f41c64d62dd'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:27.109119 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-c0062cbd-3ba7-4dbe-8cfc-b1fe65ee2edf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7286139a175841f9a6506f41c64d62dd'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:27.132376 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-c0062cbd-3ba7-4dbe-8cfc-b1fe65ee2edf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7286139a175841f9a6506f41c64d62dd'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:27.132376 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-c0062cbd-3ba7-4dbe-8cfc-b1fe65ee2edf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7286139a175841f9a6506f41c64d62dd'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:27.147448 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1024/2033] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:14:26 2022] DELETE /identity/v3/users/7286139a175841f9a6506f41c64d62dd => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:27.548597 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cba49f-3ba6-4bfa-97c3-570d4b70d23e admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:27.550084 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-c3cba49f-3ba6-4bfa-97c3-570d4b70d23e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:27.565012 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cba49f-3ba6-4bfa-97c3-570d4b70d23e admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:27.632800 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b55dcf-a5f6-4a47-8946-c829447f9638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:27.633453 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b55dcf-a5f6-4a47-8946-c829447f9638 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:27.633737 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b55dcf-a5f6-4a47-8946-c829447f9638 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:27.634010 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b55dcf-a5f6-4a47-8946-c829447f9638 admin admin] PATH_INFO: `/v3/projects/4297472c3efd4f67a08ed29c3e81d8d9` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:27.635746 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b55dcf-a5f6-4a47-8946-c829447f9638 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4297472c3efd4f67a08ed29c3e81d8d9)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:27.637804 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b55dcf-a5f6-4a47-8946-c829447f9638 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:27.691678 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-a0b55dcf-a5f6-4a47-8946-c829447f9638 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4297472c3efd4f67a08ed29c3e81d8d9', 'request_id': 'req-a0b55dcf-a5f6-4a47-8946-c829447f9638'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:27.698518 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-a0b55dcf-a5f6-4a47-8946-c829447f9638 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4297472c3efd4f67a08ed29c3e81d8d9', 'request_id': 'req-a0b55dcf-a5f6-4a47-8946-c829447f9638'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:27.705681 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-a0b55dcf-a5f6-4a47-8946-c829447f9638 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4297472c3efd4f67a08ed29c3e81d8d9', 'request_id': 'req-a0b55dcf-a5f6-4a47-8946-c829447f9638'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:27.717253 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-a0b55dcf-a5f6-4a47-8946-c829447f9638 admin admin] The token cache is being invalidate because project 4297472c3efd4f67a08ed29c3e81d8d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81661) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:14:27.717573 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-a0b55dcf-a5f6-4a47-8946-c829447f9638 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:27.718345 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-a0b55dcf-a5f6-4a47-8946-c829447f9638 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:27.720051 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1010/2034] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:14:27 2022] DELETE /identity/v3/projects/4297472c3efd4f67a08ed29c3e81d8d9 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:27.727028 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f891ca99-79b4-4aea-85ad-786973d0d3cd None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:27.727951 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f891ca99-79b4-4aea-85ad-786973d0d3cd None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:27.728248 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f891ca99-79b4-4aea-85ad-786973d0d3cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:27.762550 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-f891ca99-79b4-4aea-85ad-786973d0d3cd None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:27.763869 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-f891ca99-79b4-4aea-85ad-786973d0d3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:27.845655 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1025/2035] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:27.853494 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e40e0-30d7-4229-9ed6-0553ca7661c0 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:27.853779 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e40e0-30d7-4229-9ed6-0553ca7661c0 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:27.854074 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e40e0-30d7-4229-9ed6-0553ca7661c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:27.874966 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-976e40e0-30d7-4229-9ed6-0553ca7661c0 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:27.876355 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-976e40e0-30d7-4229-9ed6-0553ca7661c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:27.884875 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1011/2036] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:27.889912 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f891ca99-79b4-4aea-85ad-786973d0d3cd None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:27.904232 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f891ca99-79b4-4aea-85ad-786973d0d3cd None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:27.910098 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed507537-6071-457f-b628-bd43caa31f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:27.911020 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed507537-6071-457f-b628-bd43caa31f54 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:27.911296 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed507537-6071-457f-b628-bd43caa31f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:27.911566 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed507537-6071-457f-b628-bd43caa31f54 admin admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:27.912267 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed507537-6071-457f-b628-bd43caa31f54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:14:27.912641 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed507537-6071-457f-b628-bd43caa31f54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:27.914598 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed507537-6071-457f-b628-bd43caa31f54 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:27.929564 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1026/2037] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:14:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:14:27.933834 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976e40e0-30d7-4229-9ed6-0553ca7661c0 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:27.944093 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976e40e0-30d7-4229-9ed6-0553ca7661c0 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:27.949398 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1644db0-bd98-427e-9fde-d98f324dbe36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:27.950089 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1644db0-bd98-427e-9fde-d98f324dbe36 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:27.950456 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1644db0-bd98-427e-9fde-d98f324dbe36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:27.950793 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1644db0-bd98-427e-9fde-d98f324dbe36 admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:27.951502 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1644db0-bd98-427e-9fde-d98f324dbe36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:27.954520 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1644db0-bd98-427e-9fde-d98f324dbe36 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:27.987250 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1012/2038] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:14:27 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:14:27.988595 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed507537-6071-457f-b628-bd43caa31f54 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:27.997799 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:28.005080 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed507537-6071-457f-b628-bd43caa31f54 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:28.069737 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db646fec-a280-4ae3-b3d6-ed4b93fd3113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:28.070273 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db646fec-a280-4ae3-b3d6-ed4b93fd3113 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:28.070550 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db646fec-a280-4ae3-b3d6-ed4b93fd3113 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:28.070823 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db646fec-a280-4ae3-b3d6-ed4b93fd3113 admin admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:28.071561 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db646fec-a280-4ae3-b3d6-ed4b93fd3113 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:28.073936 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db646fec-a280-4ae3-b3d6-ed4b93fd3113 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:28.097292 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1027/2039] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:14:27 2022] POST /identity/v3/users => generated 292 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 01:14:28.101558 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1644db0-bd98-427e-9fde-d98f324dbe36 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:28.112389 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1644db0-bd98-427e-9fde-d98f324dbe36 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:28.117072 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c5b68e-4dab-4087-8a72-6bcd7812b66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:28.117669 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c5b68e-4dab-4087-8a72-6bcd7812b66a admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:28.117948 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c5b68e-4dab-4087-8a72-6bcd7812b66a admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:28.118243 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c5b68e-4dab-4087-8a72-6bcd7812b66a admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:28.118906 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c5b68e-4dab-4087-8a72-6bcd7812b66a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:28.120537 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c5b68e-4dab-4087-8a72-6bcd7812b66a admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:28.130900 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1013/2040] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:14:28.135611 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db646fec-a280-4ae3-b3d6-ed4b93fd3113 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:28.146681 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db646fec-a280-4ae3-b3d6-ed4b93fd3113 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:28.152125 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b9a014-ef37-43cb-8834-98a928db5f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:28.152738 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9a014-ef37-43cb-8834-98a928db5f0a admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:28.153015 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9a014-ef37-43cb-8834-98a928db5f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:28.153286 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9a014-ef37-43cb-8834-98a928db5f0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:28.154249 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b9a014-ef37-43cb-8834-98a928db5f0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:28.157010 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b9a014-ef37-43cb-8834-98a928db5f0a admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:28.169483 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1028/2041] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:14:28.173161 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c5b68e-4dab-4087-8a72-6bcd7812b66a admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:28.183113 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c5b68e-4dab-4087-8a72-6bcd7812b66a admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:28.188476 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1416dacd-e783-4b89-8aa1-1bc7c3755e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:28.189124 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1416dacd-e783-4b89-8aa1-1bc7c3755e42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:28.189405 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1416dacd-e783-4b89-8aa1-1bc7c3755e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:28.189679 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1416dacd-e783-4b89-8aa1-1bc7c3755e42 admin admin] PATH_INFO: `/v3/projects/c1f1fc48e15146cd8232dd039fea0b1f/users/6ace47df522b4cee9237d7c8f85a23ee/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:28.204728 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1416dacd-e783-4b89-8aa1-1bc7c3755e42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1f1fc48e15146cd8232dd039fea0b1f, user_id=6ace47df522b4cee9237d7c8f85a23ee, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:28.207430 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1416dacd-e783-4b89-8aa1-1bc7c3755e42 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:28.221341 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1014/2042] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:14:28 2022] PUT /identity/v3/projects/c1f1fc48e15146cd8232dd039fea0b1f/users/6ace47df522b4cee9237d7c8f85a23ee/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:28.231924 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5674fd-2c9c-4aeb-8e13-7bba8e61d6b7 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:28.232292 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5674fd-2c9c-4aeb-8e13-7bba8e61d6b7 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:28.232616 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5674fd-2c9c-4aeb-8e13-7bba8e61d6b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:28.268159 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-4a5674fd-2c9c-4aeb-8e13-7bba8e61d6b7 None None] MFA Rules not processed for user `6ace47df522b4cee9237d7c8f85a23ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:28.269393 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-4a5674fd-2c9c-4aeb-8e13-7bba8e61d6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:28.339718 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1029/2043] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:14:28.344361 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1416dacd-e783-4b89-8aa1-1bc7c3755e42 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:28.350596 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:INSERT=4,keystone:DELETE=10,keystone:SELECT=86 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:28.426056 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1416dacd-e783-4b89-8aa1-1bc7c3755e42 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:28.431008 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7bbbf0-6578-4b6e-ae5f-f158adfd21b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:28.431617 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7bbbf0-6578-4b6e-ae5f-f158adfd21b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:28.431901 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7bbbf0-6578-4b6e-ae5f-f158adfd21b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:28.432202 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7bbbf0-6578-4b6e-ae5f-f158adfd21b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:28.432880 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7bbbf0-6578-4b6e-ae5f-f158adfd21b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:28.434834 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7bbbf0-6578-4b6e-ae5f-f158adfd21b4 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:28.466020 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1015/2044] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:14:28 2022] POST /identity/v3/projects => generated 385 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:14:28.471584 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5674fd-2c9c-4aeb-8e13-7bba8e61d6b7 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:28.481485 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5674fd-2c9c-4aeb-8e13-7bba8e61d6b7 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:28.544984 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502ff493-aa66-4322-9978-7e06a5bf44f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:28.546095 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ff493-aa66-4322-9978-7e06a5bf44f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:28.546512 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ff493-aa66-4322-9978-7e06a5bf44f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:28.546865 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ff493-aa66-4322-9978-7e06a5bf44f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:28.547714 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502ff493-aa66-4322-9978-7e06a5bf44f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:28.549689 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502ff493-aa66-4322-9978-7e06a5bf44f9 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:28.567470 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:28.572745 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1030/2045] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:14:28 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 01:14:28.576835 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7bbbf0-6578-4b6e-ae5f-f158adfd21b4 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:28.593091 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7bbbf0-6578-4b6e-ae5f-f158adfd21b4 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:28.598146 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea28cfc-430c-4360-b01f-051ab6e526d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:28.598752 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea28cfc-430c-4360-b01f-051ab6e526d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:28.599041 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea28cfc-430c-4360-b01f-051ab6e526d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:28.599301 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea28cfc-430c-4360-b01f-051ab6e526d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:28.599936 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea28cfc-430c-4360-b01f-051ab6e526d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:28.601578 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea28cfc-430c-4360-b01f-051ab6e526d3 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:28.616221 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1016/2046] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 01:14:28.617939 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502ff493-aa66-4322-9978-7e06a5bf44f9 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:28.635759 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502ff493-aa66-4322-9978-7e06a5bf44f9 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:28.646408 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a1f2d-b948-4337-9005-182f2a51d1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:28.647027 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a1f2d-b948-4337-9005-182f2a51d1ef admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:28.647305 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a1f2d-b948-4337-9005-182f2a51d1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:28.647570 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a1f2d-b948-4337-9005-182f2a51d1ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:28.648250 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7a1f2d-b948-4337-9005-182f2a51d1ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:28.649874 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7a1f2d-b948-4337-9005-182f2a51d1ef admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:28.660450 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1031/2047] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 01:14:28.664947 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea28cfc-430c-4360-b01f-051ab6e526d3 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:28.676626 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea28cfc-430c-4360-b01f-051ab6e526d3 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:28.681625 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94880ac-a40e-43b7-ac15-dc37cdbed6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:28.682320 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94880ac-a40e-43b7-ac15-dc37cdbed6fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:28.682918 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94880ac-a40e-43b7-ac15-dc37cdbed6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:28.683153 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94880ac-a40e-43b7-ac15-dc37cdbed6fc admin admin] PATH_INFO: `/v3/projects/fe955ea6c89a4414a32e35cc295ae303/users/32704f34831a43d494174a045e938ab3/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:28.702179 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94880ac-a40e-43b7-ac15-dc37cdbed6fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe955ea6c89a4414a32e35cc295ae303, user_id=32704f34831a43d494174a045e938ab3, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:28.707830 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94880ac-a40e-43b7-ac15-dc37cdbed6fc admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:28.721664 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1017/2048] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:14:28 2022] PUT /identity/v3/projects/fe955ea6c89a4414a32e35cc295ae303/users/32704f34831a43d494174a045e938ab3/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:28.733766 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d609d4-4039-41d4-b695-15f782b4a4a4 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:28.734626 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d609d4-4039-41d4-b695-15f782b4a4a4 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:28.735093 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d609d4-4039-41d4-b695-15f782b4a4a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:28.779709 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-61d609d4-4039-41d4-b695-15f782b4a4a4 None None] MFA Rules not processed for user `32704f34831a43d494174a045e938ab3`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:28.781722 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-61d609d4-4039-41d4-b695-15f782b4a4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:28.855741 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1032/2049] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:14:28.861276 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94880ac-a40e-43b7-ac15-dc37cdbed6fc admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:28.926290 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94880ac-a40e-43b7-ac15-dc37cdbed6fc admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:28.931097 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75113346-406d-443d-b67b-ec3e2beb0f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:28.931704 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75113346-406d-443d-b67b-ec3e2beb0f67 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:28.931984 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75113346-406d-443d-b67b-ec3e2beb0f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:28.932289 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75113346-406d-443d-b67b-ec3e2beb0f67 admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:28.932919 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75113346-406d-443d-b67b-ec3e2beb0f67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:28.934833 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75113346-406d-443d-b67b-ec3e2beb0f67 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:28.961313 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1018/2050] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:14:28 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:14:28.965369 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d609d4-4039-41d4-b695-15f782b4a4a4 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:28.976957 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d609d4-4039-41d4-b695-15f782b4a4a4 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:29.040386 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b49290-3cd0-40c0-bd12-ef76cdb02222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:29.040996 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b49290-3cd0-40c0-bd12-ef76cdb02222 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:29.041273 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b49290-3cd0-40c0-bd12-ef76cdb02222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:29.041544 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b49290-3cd0-40c0-bd12-ef76cdb02222 admin admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:29.042189 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b49290-3cd0-40c0-bd12-ef76cdb02222 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:29.044407 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b49290-3cd0-40c0-bd12-ef76cdb02222 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:29.067469 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1033/2051] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:14:28 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:14:29.071613 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75113346-406d-443d-b67b-ec3e2beb0f67 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:29.082310 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75113346-406d-443d-b67b-ec3e2beb0f67 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:29.087052 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3977f88b-34fd-4f05-bb20-ebedcc186bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:29.087651 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3977f88b-34fd-4f05-bb20-ebedcc186bf8 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:29.087930 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3977f88b-34fd-4f05-bb20-ebedcc186bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:29.088240 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3977f88b-34fd-4f05-bb20-ebedcc186bf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:29.088879 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3977f88b-34fd-4f05-bb20-ebedcc186bf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:29.090474 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3977f88b-34fd-4f05-bb20-ebedcc186bf8 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:29.101607 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1019/2052] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:14:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:14:29.105727 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b49290-3cd0-40c0-bd12-ef76cdb02222 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:29.116974 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b49290-3cd0-40c0-bd12-ef76cdb02222 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:29.121564 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef7419-c7e2-407b-bec4-d1106976d623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:29.122202 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef7419-c7e2-407b-bec4-d1106976d623 admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:29.122478 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef7419-c7e2-407b-bec4-d1106976d623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:29.122781 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef7419-c7e2-407b-bec4-d1106976d623 admin admin] PATH_INFO: `/v3/projects/d5321991bcbb442bb757cddf7a2914f1/users/3b18afe1160f417e8ca28f4a5fb7b9b5/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:29.137803 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ef7419-c7e2-407b-bec4-d1106976d623 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5321991bcbb442bb757cddf7a2914f1, user_id=3b18afe1160f417e8ca28f4a5fb7b9b5, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:29.140893 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ef7419-c7e2-407b-bec4-d1106976d623 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:29.153671 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1034/2053] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:14:29 2022] PUT /identity/v3/projects/d5321991bcbb442bb757cddf7a2914f1/users/3b18afe1160f417e8ca28f4a5fb7b9b5/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:29.158109 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3977f88b-34fd-4f05-bb20-ebedcc186bf8 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:29.168108 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3977f88b-34fd-4f05-bb20-ebedcc186bf8 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:29.194601 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:29.225529 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672b035b-2b25-4703-9f91-9563569ec604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:29.226136 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b035b-2b25-4703-9f91-9563569ec604 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:29.226416 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b035b-2b25-4703-9f91-9563569ec604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:29.226686 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b035b-2b25-4703-9f91-9563569ec604 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:29.227315 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672b035b-2b25-4703-9f91-9563569ec604 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:29.228981 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672b035b-2b25-4703-9f91-9563569ec604 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:29.240710 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1020/2054] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:14:29 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:14:29.244953 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef7419-c7e2-407b-bec4-d1106976d623 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:29.255823 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef7419-c7e2-407b-bec4-d1106976d623 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:29.260722 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7392cb06-884c-4932-adcd-2bf7e6b508ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:29.261390 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7392cb06-884c-4932-adcd-2bf7e6b508ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:29.261667 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7392cb06-884c-4932-adcd-2bf7e6b508ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:29.261934 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7392cb06-884c-4932-adcd-2bf7e6b508ef admin admin] PATH_INFO: `/v3/projects/d5321991bcbb442bb757cddf7a2914f1/users/3b18afe1160f417e8ca28f4a5fb7b9b5/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:29.263895 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7392cb06-884c-4932-adcd-2bf7e6b508ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5321991bcbb442bb757cddf7a2914f1, user_id=3b18afe1160f417e8ca28f4a5fb7b9b5, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:29.266871 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7392cb06-884c-4932-adcd-2bf7e6b508ef admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:29.277262 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1035/2055] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:14:29 2022] PUT /identity/v3/projects/d5321991bcbb442bb757cddf7a2914f1/users/3b18afe1160f417e8ca28f4a5fb7b9b5/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:29.288422 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de7ff7-6a11-463a-bcbc-fa1626ecbc44 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:29.288711 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de7ff7-6a11-463a-bcbc-fa1626ecbc44 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:29.288984 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de7ff7-6a11-463a-bcbc-fa1626ecbc44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:29.316870 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-15de7ff7-6a11-463a-bcbc-fa1626ecbc44 None None] MFA Rules not processed for user `3b18afe1160f417e8ca28f4a5fb7b9b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:29.318070 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-15de7ff7-6a11-463a-bcbc-fa1626ecbc44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:29.385932 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1021/2056] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:29 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:29.397931 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7392cb06-884c-4932-adcd-2bf7e6b508ef admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:29.399654 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-7392cb06-884c-4932-adcd-2bf7e6b508ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:29.441499 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:29.472280 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7392cb06-884c-4932-adcd-2bf7e6b508ef admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:29.476520 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838f1f20-d11d-4323-8da7-adf2a1eb204f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:29.477122 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f1f20-d11d-4323-8da7-adf2a1eb204f service cinder] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:29.477403 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f1f20-d11d-4323-8da7-adf2a1eb204f service cinder] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:29.477671 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f1f20-d11d-4323-8da7-adf2a1eb204f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:29.544899 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838f1f20-d11d-4323-8da7-adf2a1eb204f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:29.547119 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838f1f20-d11d-4323-8da7-adf2a1eb204f service cinder] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:29.560867 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1036/2057] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:14:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:14:32.138814 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15de7ff7-6a11-463a-bcbc-fa1626ecbc44 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:32.149495 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15de7ff7-6a11-463a-bcbc-fa1626ecbc44 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:32.153284 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d3d58-68b3-4415-9bc8-0a8e24213456 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:32.153972 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d3d58-68b3-4415-9bc8-0a8e24213456 service cinder] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:32.154351 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d3d58-68b3-4415-9bc8-0a8e24213456 service cinder] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:32.154699 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d3d58-68b3-4415-9bc8-0a8e24213456 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:32.157134 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-0f4d3d58-68b3-4415-9bc8-0a8e24213456 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:32.221227 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4d3d58-68b3-4415-9bc8-0a8e24213456 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:32.222672 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4d3d58-68b3-4415-9bc8-0a8e24213456 service cinder] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:32.232064 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1022/2058] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:14:32 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:14:33.267944 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838f1f20-d11d-4323-8da7-adf2a1eb204f service cinder] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:33.269454 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-838f1f20-d11d-4323-8da7-adf2a1eb204f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:33.281287 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838f1f20-d11d-4323-8da7-adf2a1eb204f service cinder] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:33.285057 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0bb69f-9216-4b81-985f-a0002280af16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:33.285660 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0bb69f-9216-4b81-985f-a0002280af16 service cinder] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:33.285938 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0bb69f-9216-4b81-985f-a0002280af16 service cinder] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:33.286215 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0bb69f-9216-4b81-985f-a0002280af16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:33.352745 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0bb69f-9216-4b81-985f-a0002280af16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:33.354431 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0bb69f-9216-4b81-985f-a0002280af16 service cinder] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:33.364104 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1037/2059] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:14:33 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:14:36.203034 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d3d58-68b3-4415-9bc8-0a8e24213456 service cinder] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:36.211913 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d3d58-68b3-4415-9bc8-0a8e24213456 service cinder] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:36.215547 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71bab63-a6e5-48c5-9200-d845ad360c09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:36.216165 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71bab63-a6e5-48c5-9200-d845ad360c09 service cinder] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:36.216446 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71bab63-a6e5-48c5-9200-d845ad360c09 service cinder] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:36.216718 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71bab63-a6e5-48c5-9200-d845ad360c09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:36.225895 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71bab63-a6e5-48c5-9200-d845ad360c09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:36.227337 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71bab63-a6e5-48c5-9200-d845ad360c09 service cinder] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:36.237858 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1023/2060] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:14:36 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:14:37.718694 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2d585-d94f-4e97-baad-bf258290f0b5 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:37.719150 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2d585-d94f-4e97-baad-bf258290f0b5 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:37.719450 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2d585-d94f-4e97-baad-bf258290f0b5 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:37.721096 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1038/2061] 10.222.0.42 () {54 vars in 921 bytes} [Thu Sep 8 01:14:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:14:37.751716 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f327b66-d08e-4a51-8036-86e8d94c9541 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:37.752268 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f327b66-d08e-4a51-8036-86e8d94c9541 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:37.752362 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f327b66-d08e-4a51-8036-86e8d94c9541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:37.783508 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-8f327b66-d08e-4a51-8036-86e8d94c9541 None None] MFA Rules not processed for user `301d30bbd3de4cc69e96bb12729442c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:37.784989 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-8f327b66-d08e-4a51-8036-86e8d94c9541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:37.825956 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:37.870214 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1024/2062] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Sep 8 01:14:37 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:14:38.068826 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a249c2e3-aa5a-4a95-a89f-b401da9565e8 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:38.069456 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a249c2e3-aa5a-4a95-a89f-b401da9565e8 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:38.070270 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a249c2e3-aa5a-4a95-a89f-b401da9565e8 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:38.071888 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1039/2063] 10.222.0.42 () {54 vars in 961 bytes} [Thu Sep 8 01:14:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 01:14:38.088957 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237b571-3014-428d-a04a-8f26ab8fa9ae None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:38.089432 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237b571-3014-428d-a04a-8f26ab8fa9ae None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:38.089671 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237b571-3014-428d-a04a-8f26ab8fa9ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:38.111865 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-6237b571-3014-428d-a04a-8f26ab8fa9ae None None] MFA Rules not processed for user `301d30bbd3de4cc69e96bb12729442c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:38.113626 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-6237b571-3014-428d-a04a-8f26ab8fa9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:38.123576 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1025/2064] 10.222.0.42 () {58 vars in 1058 bytes} [Thu Sep 8 01:14:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:38.132798 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a249c2e3-aa5a-4a95-a89f-b401da9565e8 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:38.166330 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a249c2e3-aa5a-4a95-a89f-b401da9565e8 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:38.178006 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e3a59a-fddf-481c-88cf-521cb6722b61 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '301d30bbd3de4cc69e96bb12729442c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:38.178714 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3a59a-fddf-481c-88cf-521cb6722b61 service placement] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:38.179046 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3a59a-fddf-481c-88cf-521cb6722b61 service placement] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:38.179385 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3a59a-fddf-481c-88cf-521cb6722b61 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:38.192067 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e3a59a-fddf-481c-88cf-521cb6722b61 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:38.193993 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e3a59a-fddf-481c-88cf-521cb6722b61 service placement] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:38.204408 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1040/2065] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:14:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:14:38.617477 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6237b571-3014-428d-a04a-8f26ab8fa9ae None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:38.618916 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-6237b571-3014-428d-a04a-8f26ab8fa9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:38.693522 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6237b571-3014-428d-a04a-8f26ab8fa9ae None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:38.702398 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a3c5d3-2d8f-41f5-8042-74fa7e0a0af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:38.703115 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3c5d3-2d8f-41f5-8042-74fa7e0a0af0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:38.703404 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3c5d3-2d8f-41f5-8042-74fa7e0a0af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:38.703683 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3c5d3-2d8f-41f5-8042-74fa7e0a0af0 admin admin] PATH_INFO: `/v3/users/a833d3dfb459415886b8798cd3800237` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:38.704851 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a3c5d3-2d8f-41f5-8042-74fa7e0a0af0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a833d3dfb459415886b8798cd3800237)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:38.706807 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a3c5d3-2d8f-41f5-8042-74fa7e0a0af0 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:38.764939 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-72a3c5d3-2d8f-41f5-8042-74fa7e0a0af0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a833d3dfb459415886b8798cd3800237'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:38.776814 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-72a3c5d3-2d8f-41f5-8042-74fa7e0a0af0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a833d3dfb459415886b8798cd3800237'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:38.784410 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-72a3c5d3-2d8f-41f5-8042-74fa7e0a0af0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a833d3dfb459415886b8798cd3800237'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:38.785132 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-72a3c5d3-2d8f-41f5-8042-74fa7e0a0af0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a833d3dfb459415886b8798cd3800237'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:38.795929 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-72a3c5d3-2d8f-41f5-8042-74fa7e0a0af0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a833d3dfb459415886b8798cd3800237'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:38.802780 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1026/2066] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:14:38 2022] DELETE /identity/v3/users/a833d3dfb459415886b8798cd3800237 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:39.011598 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e3a59a-fddf-481c-88cf-521cb6722b61 service placement] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:39.031392 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e3a59a-fddf-481c-88cf-521cb6722b61 service placement] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:39.109237 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef3435-eb62-4e58-bf37-840771eb3567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:39.109923 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef3435-eb62-4e58-bf37-840771eb3567 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:39.110211 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef3435-eb62-4e58-bf37-840771eb3567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:39.110489 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef3435-eb62-4e58-bf37-840771eb3567 admin admin] PATH_INFO: `/v3/projects/98839645cd3c4f5eb0339cb6c6304d2d` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:39.112090 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebef3435-eb62-4e58-bf37-840771eb3567 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98839645cd3c4f5eb0339cb6c6304d2d)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:39.114247 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebef3435-eb62-4e58-bf37-840771eb3567 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:39.128023 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:39.171660 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-ebef3435-eb62-4e58-bf37-840771eb3567 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98839645cd3c4f5eb0339cb6c6304d2d', 'request_id': 'req-ebef3435-eb62-4e58-bf37-840771eb3567'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:39.177862 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-ebef3435-eb62-4e58-bf37-840771eb3567 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98839645cd3c4f5eb0339cb6c6304d2d', 'request_id': 'req-ebef3435-eb62-4e58-bf37-840771eb3567'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:39.184447 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-ebef3435-eb62-4e58-bf37-840771eb3567 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98839645cd3c4f5eb0339cb6c6304d2d', 'request_id': 'req-ebef3435-eb62-4e58-bf37-840771eb3567'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:39.198623 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-ebef3435-eb62-4e58-bf37-840771eb3567 admin admin] The token cache is being invalidate because project 98839645cd3c4f5eb0339cb6c6304d2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:14:39.199128 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-ebef3435-eb62-4e58-bf37-840771eb3567 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:39.199874 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-ebef3435-eb62-4e58-bf37-840771eb3567 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:39.206847 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1041/2067] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:14:39 2022] DELETE /identity/v3/projects/98839645cd3c4f5eb0339cb6c6304d2d => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:39.210723 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a3c5d3-2d8f-41f5-8042-74fa7e0a0af0 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:39.212180 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-72a3c5d3-2d8f-41f5-8042-74fa7e0a0af0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:39.225629 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a3c5d3-2d8f-41f5-8042-74fa7e0a0af0 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:39.265561 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:39.286021 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05620aa8-66dd-4123-aff6-080c42dc01ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:39.286658 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05620aa8-66dd-4123-aff6-080c42dc01ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:39.286940 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05620aa8-66dd-4123-aff6-080c42dc01ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:39.287213 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05620aa8-66dd-4123-aff6-080c42dc01ca admin admin] PATH_INFO: `/v3/users/79ac51f5326f41d1872bf70cd7bb2981` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:39.288370 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05620aa8-66dd-4123-aff6-080c42dc01ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=79ac51f5326f41d1872bf70cd7bb2981)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:39.290339 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05620aa8-66dd-4123-aff6-080c42dc01ca admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:39.335605 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-05620aa8-66dd-4123-aff6-080c42dc01ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79ac51f5326f41d1872bf70cd7bb2981'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:39.347992 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-05620aa8-66dd-4123-aff6-080c42dc01ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79ac51f5326f41d1872bf70cd7bb2981'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:39.354861 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-05620aa8-66dd-4123-aff6-080c42dc01ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79ac51f5326f41d1872bf70cd7bb2981'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:39.355523 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-05620aa8-66dd-4123-aff6-080c42dc01ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79ac51f5326f41d1872bf70cd7bb2981'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:39.362820 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-05620aa8-66dd-4123-aff6-080c42dc01ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79ac51f5326f41d1872bf70cd7bb2981'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:39.368833 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1027/2068] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:14:39 2022] DELETE /identity/v3/users/79ac51f5326f41d1872bf70cd7bb2981 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:39.540317 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef3435-eb62-4e58-bf37-840771eb3567 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:39.541806 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-ebef3435-eb62-4e58-bf37-840771eb3567 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:39.559936 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef3435-eb62-4e58-bf37-840771eb3567 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:39.620882 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c6ac22-1c7f-4bc3-8541-6707514b6d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:39.621525 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c6ac22-1c7f-4bc3-8541-6707514b6d4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:39.622368 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c6ac22-1c7f-4bc3-8541-6707514b6d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:39.622648 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c6ac22-1c7f-4bc3-8541-6707514b6d4c admin admin] PATH_INFO: `/v3/projects/0308978cd8c142c6ad93e27155824889` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:39.624472 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c6ac22-1c7f-4bc3-8541-6707514b6d4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0308978cd8c142c6ad93e27155824889)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:39.626428 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c6ac22-1c7f-4bc3-8541-6707514b6d4c admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:39.673389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-18c6ac22-1c7f-4bc3-8541-6707514b6d4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0308978cd8c142c6ad93e27155824889', 'request_id': 'req-18c6ac22-1c7f-4bc3-8541-6707514b6d4c'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:39.679494 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-18c6ac22-1c7f-4bc3-8541-6707514b6d4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0308978cd8c142c6ad93e27155824889', 'request_id': 'req-18c6ac22-1c7f-4bc3-8541-6707514b6d4c'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:39.685502 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-18c6ac22-1c7f-4bc3-8541-6707514b6d4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0308978cd8c142c6ad93e27155824889', 'request_id': 'req-18c6ac22-1c7f-4bc3-8541-6707514b6d4c'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:39.698404 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-18c6ac22-1c7f-4bc3-8541-6707514b6d4c admin admin] The token cache is being invalidate because project 0308978cd8c142c6ad93e27155824889 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:14:39.698736 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-18c6ac22-1c7f-4bc3-8541-6707514b6d4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:39.699326 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-18c6ac22-1c7f-4bc3-8541-6707514b6d4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:39.700847 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1042/2069] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:14:39 2022] DELETE /identity/v3/projects/0308978cd8c142c6ad93e27155824889 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:14:39.706816 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0cdb2b-37eb-4447-aa95-073e0de6315a None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:39.707102 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0cdb2b-37eb-4447-aa95-073e0de6315a None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:39.707399 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0cdb2b-37eb-4447-aa95-073e0de6315a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:39.727363 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-ba0cdb2b-37eb-4447-aa95-073e0de6315a None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:39.728615 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-ba0cdb2b-37eb-4447-aa95-073e0de6315a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:39.788849 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1028/2070] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:39.798111 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c355aaa7-d16a-4055-ac3a-7a479a94bf3a None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:39.798397 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c355aaa7-d16a-4055-ac3a-7a479a94bf3a None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:39.798672 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c355aaa7-d16a-4055-ac3a-7a479a94bf3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:39.819561 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-c355aaa7-d16a-4055-ac3a-7a479a94bf3a None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:39.821008 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-c355aaa7-d16a-4055-ac3a-7a479a94bf3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:39.829149 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1043/2071] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:39.833908 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0cdb2b-37eb-4447-aa95-073e0de6315a None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:39.844483 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0cdb2b-37eb-4447-aa95-073e0de6315a None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:39.849092 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2794dcf1-72f1-4bae-a730-361f3bb863a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:39.849698 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2794dcf1-72f1-4bae-a730-361f3bb863a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:39.849978 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2794dcf1-72f1-4bae-a730-361f3bb863a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:39.850250 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2794dcf1-72f1-4bae-a730-361f3bb863a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:39.850903 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2794dcf1-72f1-4bae-a730-361f3bb863a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:14:39.851208 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2794dcf1-72f1-4bae-a730-361f3bb863a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:39.852849 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2794dcf1-72f1-4bae-a730-361f3bb863a9 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:39.868178 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1029/2072] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:14:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:14:39.869906 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c355aaa7-d16a-4055-ac3a-7a479a94bf3a None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:39.880123 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c355aaa7-d16a-4055-ac3a-7a479a94bf3a None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:39.885205 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e044b711-ceda-4d84-9890-575673da2754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:39.885813 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e044b711-ceda-4d84-9890-575673da2754 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:39.886092 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e044b711-ceda-4d84-9890-575673da2754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:39.886366 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e044b711-ceda-4d84-9890-575673da2754 admin admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:39.887023 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e044b711-ceda-4d84-9890-575673da2754 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:39.889134 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e044b711-ceda-4d84-9890-575673da2754 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:39.915493 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1044/2073] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:14:39 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:14:39.919613 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2794dcf1-72f1-4bae-a730-361f3bb863a9 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:39.930056 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2794dcf1-72f1-4bae-a730-361f3bb863a9 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:39.972645 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:39.986459 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd00304-2dca-4fa4-b49d-cb13cb31074c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:39.987055 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd00304-2dca-4fa4-b49d-cb13cb31074c admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:39.987337 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd00304-2dca-4fa4-b49d-cb13cb31074c admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:39.987611 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd00304-2dca-4fa4-b49d-cb13cb31074c admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:39.988273 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd00304-2dca-4fa4-b49d-cb13cb31074c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:39.990195 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd00304-2dca-4fa4-b49d-cb13cb31074c admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:40.011801 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1030/2074] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:14:39 2022] POST /identity/v3/users => generated 286 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:14:40.015966 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e044b711-ceda-4d84-9890-575673da2754 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:40.021607 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:40.027949 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e044b711-ceda-4d84-9890-575673da2754 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:40.033853 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910ba1bd-2f1b-4a1a-8db5-e21fd75f2f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:40.034452 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ba1bd-2f1b-4a1a-8db5-e21fd75f2f39 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:40.034736 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ba1bd-2f1b-4a1a-8db5-e21fd75f2f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:40.035011 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ba1bd-2f1b-4a1a-8db5-e21fd75f2f39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:40.035659 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910ba1bd-2f1b-4a1a-8db5-e21fd75f2f39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:40.037348 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910ba1bd-2f1b-4a1a-8db5-e21fd75f2f39 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:40.047771 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1045/2075] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:14:40.051845 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd00304-2dca-4fa4-b49d-cb13cb31074c admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:40.062705 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd00304-2dca-4fa4-b49d-cb13cb31074c admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:40.067312 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0862837c-bc54-4616-b87f-4c7b2f55d161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:40.067896 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0862837c-bc54-4616-b87f-4c7b2f55d161 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:40.068272 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0862837c-bc54-4616-b87f-4c7b2f55d161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:40.068548 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0862837c-bc54-4616-b87f-4c7b2f55d161 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:40.069177 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0862837c-bc54-4616-b87f-4c7b2f55d161 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:40.070756 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0862837c-bc54-4616-b87f-4c7b2f55d161 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:40.081526 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1031/2076] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:14:40.085474 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910ba1bd-2f1b-4a1a-8db5-e21fd75f2f39 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:40.095453 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910ba1bd-2f1b-4a1a-8db5-e21fd75f2f39 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:40.100041 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f947725-8a15-45d2-90ce-bd490d3da1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:40.100698 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f947725-8a15-45d2-90ce-bd490d3da1a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:40.100972 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f947725-8a15-45d2-90ce-bd490d3da1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:40.101242 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f947725-8a15-45d2-90ce-bd490d3da1a2 admin admin] PATH_INFO: `/v3/projects/58cdd426167d4bef8c63a9ebfa7ee33f/users/405e9b8e9a9341b487153c5b48e7d169/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:40.116262 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f947725-8a15-45d2-90ce-bd490d3da1a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58cdd426167d4bef8c63a9ebfa7ee33f, user_id=405e9b8e9a9341b487153c5b48e7d169, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:40.118910 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f947725-8a15-45d2-90ce-bd490d3da1a2 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:40.132793 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1046/2077] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:14:40 2022] PUT /identity/v3/projects/58cdd426167d4bef8c63a9ebfa7ee33f/users/405e9b8e9a9341b487153c5b48e7d169/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:40.143518 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b99853a-60a4-432a-9f40-c305dd30c31c None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:40.143867 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b99853a-60a4-432a-9f40-c305dd30c31c None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:40.144267 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b99853a-60a4-432a-9f40-c305dd30c31c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:40.178352 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-3b99853a-60a4-432a-9f40-c305dd30c31c None None] MFA Rules not processed for user `405e9b8e9a9341b487153c5b48e7d169`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:40.179692 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-3b99853a-60a4-432a-9f40-c305dd30c31c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:40.270664 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1032/2078] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:40.282776 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f947725-8a15-45d2-90ce-bd490d3da1a2 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:40.284825 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-6f947725-8a15-45d2-90ce-bd490d3da1a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:40.347794 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f947725-8a15-45d2-90ce-bd490d3da1a2 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:40.351435 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28659763-c131-4ec6-b994-c295bdc6f6a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:40.352031 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28659763-c131-4ec6-b994-c295bdc6f6a4 service cinder] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:40.352343 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28659763-c131-4ec6-b994-c295bdc6f6a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:40.352614 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28659763-c131-4ec6-b994-c295bdc6f6a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:40.360744 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28659763-c131-4ec6-b994-c295bdc6f6a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:40.362167 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28659763-c131-4ec6-b994-c295bdc6f6a4 service cinder] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:40.371097 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1047/2079] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:14:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:14:40.394228 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b99853a-60a4-432a-9f40-c305dd30c31c None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:40.456674 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b99853a-60a4-432a-9f40-c305dd30c31c None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:40.461263 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed893c-8010-4037-a395-169f6b4e1155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:40.461885 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed893c-8010-4037-a395-169f6b4e1155 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:40.462164 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed893c-8010-4037-a395-169f6b4e1155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:40.462438 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed893c-8010-4037-a395-169f6b4e1155 admin admin] PATH_INFO: `/v3/users/405e9b8e9a9341b487153c5b48e7d169` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:40.463558 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ed893c-8010-4037-a395-169f6b4e1155 admin admin] RBAC: Authorizing `identity:delete_user(user_id=405e9b8e9a9341b487153c5b48e7d169)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:40.465507 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ed893c-8010-4037-a395-169f6b4e1155 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:40.515993 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-97ed893c-8010-4037-a395-169f6b4e1155 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '405e9b8e9a9341b487153c5b48e7d169'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:40.524076 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:40.530330 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-97ed893c-8010-4037-a395-169f6b4e1155 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '405e9b8e9a9341b487153c5b48e7d169'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:40.538196 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-97ed893c-8010-4037-a395-169f6b4e1155 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '405e9b8e9a9341b487153c5b48e7d169'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:40.538903 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-97ed893c-8010-4037-a395-169f6b4e1155 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '405e9b8e9a9341b487153c5b48e7d169'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:40.546548 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-97ed893c-8010-4037-a395-169f6b4e1155 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '405e9b8e9a9341b487153c5b48e7d169'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:40.552693 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1033/2080] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:14:40 2022] DELETE /identity/v3/users/405e9b8e9a9341b487153c5b48e7d169 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:40.557258 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db44719b-002e-41f1-bb99-842374e6095b None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:40.557545 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db44719b-002e-41f1-bb99-842374e6095b None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:40.557839 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db44719b-002e-41f1-bb99-842374e6095b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:40.577474 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-db44719b-002e-41f1-bb99-842374e6095b None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:40.578673 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-db44719b-002e-41f1-bb99-842374e6095b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:40.638417 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1048/2081] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:40.648969 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed893c-8010-4037-a395-169f6b4e1155 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:40.650674 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-97ed893c-8010-4037-a395-169f6b4e1155 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:40.707929 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed893c-8010-4037-a395-169f6b4e1155 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:40.711635 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee4610e-e920-47b7-a1bc-be19d3d68e6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:40.712277 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4610e-e920-47b7-a1bc-be19d3d68e6f service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:40.712561 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4610e-e920-47b7-a1bc-be19d3d68e6f service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:40.712835 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4610e-e920-47b7-a1bc-be19d3d68e6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:40.722689 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee4610e-e920-47b7-a1bc-be19d3d68e6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:40.724179 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee4610e-e920-47b7-a1bc-be19d3d68e6f service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:40.733735 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1034/2082] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:14:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:14:41.068917 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db44719b-002e-41f1-bb99-842374e6095b None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:41.071216 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-db44719b-002e-41f1-bb99-842374e6095b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:41.085012 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db44719b-002e-41f1-bb99-842374e6095b None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:41.090198 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b932d877-c258-4d4a-bbe6-3d23877cc564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:41.090486 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b932d877-c258-4d4a-bbe6-3d23877cc564 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:41.090770 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b932d877-c258-4d4a-bbe6-3d23877cc564 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:41.091042 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b932d877-c258-4d4a-bbe6-3d23877cc564 admin admin] PATH_INFO: `/v3/projects/58cdd426167d4bef8c63a9ebfa7ee33f` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:41.092239 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b932d877-c258-4d4a-bbe6-3d23877cc564 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58cdd426167d4bef8c63a9ebfa7ee33f)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:41.094367 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b932d877-c258-4d4a-bbe6-3d23877cc564 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:41.142512 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-b932d877-c258-4d4a-bbe6-3d23877cc564 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58cdd426167d4bef8c63a9ebfa7ee33f', 'request_id': 'req-b932d877-c258-4d4a-bbe6-3d23877cc564'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:41.148529 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-b932d877-c258-4d4a-bbe6-3d23877cc564 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58cdd426167d4bef8c63a9ebfa7ee33f', 'request_id': 'req-b932d877-c258-4d4a-bbe6-3d23877cc564'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:41.154103 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-b932d877-c258-4d4a-bbe6-3d23877cc564 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58cdd426167d4bef8c63a9ebfa7ee33f', 'request_id': 'req-b932d877-c258-4d4a-bbe6-3d23877cc564'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:41.157338 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:41.168624 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-b932d877-c258-4d4a-bbe6-3d23877cc564 admin admin] The token cache is being invalidate because project 58cdd426167d4bef8c63a9ebfa7ee33f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:14:41.168913 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-b932d877-c258-4d4a-bbe6-3d23877cc564 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:41.169532 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-b932d877-c258-4d4a-bbe6-3d23877cc564 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:41.171590 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1049/2083] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:14:41 2022] DELETE /identity/v3/projects/58cdd426167d4bef8c63a9ebfa7ee33f => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:41.176564 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da1b09-e36b-401d-b1bd-fd596e47c860 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:41.177026 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da1b09-e36b-401d-b1bd-fd596e47c860 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:41.177138 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da1b09-e36b-401d-b1bd-fd596e47c860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:41.196672 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-d8da1b09-e36b-401d-b1bd-fd596e47c860 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:41.197872 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-d8da1b09-e36b-401d-b1bd-fd596e47c860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:41.278738 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1035/2084] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:41.288289 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b03c0-949f-4cce-8b0c-bc7f66957994 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:41.288703 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b03c0-949f-4cce-8b0c-bc7f66957994 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:41.288990 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b03c0-949f-4cce-8b0c-bc7f66957994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:41.309263 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-6a3b03c0-949f-4cce-8b0c-bc7f66957994 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:41.310476 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-6a3b03c0-949f-4cce-8b0c-bc7f66957994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:41.318622 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1050/2085] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:41.322600 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da1b09-e36b-401d-b1bd-fd596e47c860 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:41.334753 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da1b09-e36b-401d-b1bd-fd596e47c860 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:41.341939 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3c598b-5484-443a-b234-e0898bfcd2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:41.343424 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c598b-5484-443a-b234-e0898bfcd2e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:41.343686 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c598b-5484-443a-b234-e0898bfcd2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:41.343956 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c598b-5484-443a-b234-e0898bfcd2e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:41.344632 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3c598b-5484-443a-b234-e0898bfcd2e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:14:41.344935 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3c598b-5484-443a-b234-e0898bfcd2e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:41.346530 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3c598b-5484-443a-b234-e0898bfcd2e5 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:41.358487 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1036/2086] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:14:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:14:41.362784 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3b03c0-949f-4cce-8b0c-bc7f66957994 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:41.372229 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3b03c0-949f-4cce-8b0c-bc7f66957994 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:41.377147 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d6620c-6dca-4532-8031-f695dd8b5060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:41.377759 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d6620c-6dca-4532-8031-f695dd8b5060 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:41.378040 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d6620c-6dca-4532-8031-f695dd8b5060 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:41.378316 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d6620c-6dca-4532-8031-f695dd8b5060 admin admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:41.379037 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d6620c-6dca-4532-8031-f695dd8b5060 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:41.381000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d6620c-6dca-4532-8031-f695dd8b5060 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:41.411464 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1051/2087] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:14:41 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:14:41.415899 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3c598b-5484-443a-b234-e0898bfcd2e5 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:41.426317 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3c598b-5484-443a-b234-e0898bfcd2e5 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:41.485378 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907c415a-7597-4ca3-a094-5b8ecd10c7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:41.486011 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c415a-7597-4ca3-a094-5b8ecd10c7a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:41.486284 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c415a-7597-4ca3-a094-5b8ecd10c7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:41.486554 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c415a-7597-4ca3-a094-5b8ecd10c7a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:41.487187 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907c415a-7597-4ca3-a094-5b8ecd10c7a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:41.489138 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907c415a-7597-4ca3-a094-5b8ecd10c7a5 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:41.499884 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:41.512545 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1037/2088] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:14:41 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 01:14:41.517121 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d6620c-6dca-4532-8031-f695dd8b5060 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:41.527424 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d6620c-6dca-4532-8031-f695dd8b5060 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:41.532696 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead00535-a94d-4a5c-b27e-8e103e0245cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:41.533289 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead00535-a94d-4a5c-b27e-8e103e0245cf admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:41.533570 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead00535-a94d-4a5c-b27e-8e103e0245cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:41.533845 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead00535-a94d-4a5c-b27e-8e103e0245cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:41.534489 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead00535-a94d-4a5c-b27e-8e103e0245cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:41.536090 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead00535-a94d-4a5c-b27e-8e103e0245cf admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:41.546076 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1052/2089] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:14:41.549945 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907c415a-7597-4ca3-a094-5b8ecd10c7a5 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:41.559883 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907c415a-7597-4ca3-a094-5b8ecd10c7a5 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:41.564917 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab8c3bd-2644-4d7a-89e2-a43866e1daff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:41.565528 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab8c3bd-2644-4d7a-89e2-a43866e1daff admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:41.565825 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab8c3bd-2644-4d7a-89e2-a43866e1daff admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:41.566093 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab8c3bd-2644-4d7a-89e2-a43866e1daff admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:41.566753 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab8c3bd-2644-4d7a-89e2-a43866e1daff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:41.568371 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab8c3bd-2644-4d7a-89e2-a43866e1daff admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:41.579940 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1038/2090] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:14:41.583939 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead00535-a94d-4a5c-b27e-8e103e0245cf admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:41.594481 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead00535-a94d-4a5c-b27e-8e103e0245cf admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:41.599297 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed41a3-9ecb-4480-9875-49f32c3282dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:41.599936 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed41a3-9ecb-4480-9875-49f32c3282dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:41.600250 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed41a3-9ecb-4480-9875-49f32c3282dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:41.600530 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed41a3-9ecb-4480-9875-49f32c3282dd admin admin] PATH_INFO: `/v3/projects/322ea50d34cd4311812b0a75d89df303/users/e20e42b4418f41609cd8c3341418e56f/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:41.615134 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fed41a3-9ecb-4480-9875-49f32c3282dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=322ea50d34cd4311812b0a75d89df303, user_id=e20e42b4418f41609cd8c3341418e56f, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:41.617869 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fed41a3-9ecb-4480-9875-49f32c3282dd admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:41.631222 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1053/2091] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:14:41 2022] PUT /identity/v3/projects/322ea50d34cd4311812b0a75d89df303/users/e20e42b4418f41609cd8c3341418e56f/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:41.643088 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3235684-091d-4e81-852a-6ab25cbab808 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:41.643370 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3235684-091d-4e81-852a-6ab25cbab808 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:41.643641 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3235684-091d-4e81-852a-6ab25cbab808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:41.671256 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-f3235684-091d-4e81-852a-6ab25cbab808 None None] MFA Rules not processed for user `e20e42b4418f41609cd8c3341418e56f`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:41.672515 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-f3235684-091d-4e81-852a-6ab25cbab808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:41.736449 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1039/2092] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:41.747193 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed41a3-9ecb-4480-9875-49f32c3282dd admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:41.749477 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-1fed41a3-9ecb-4480-9875-49f32c3282dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:41.805962 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed41a3-9ecb-4480-9875-49f32c3282dd admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:41.809902 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57920543-ca60-455a-823d-40057b0a718e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:41.810505 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57920543-ca60-455a-823d-40057b0a718e service cinder] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:41.811279 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57920543-ca60-455a-823d-40057b0a718e service cinder] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:41.811548 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57920543-ca60-455a-823d-40057b0a718e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:41.819979 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57920543-ca60-455a-823d-40057b0a718e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:41.821442 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57920543-ca60-455a-823d-40057b0a718e service cinder] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:41.830749 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1054/2093] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:14:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:14:43.610933 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3235684-091d-4e81-852a-6ab25cbab808 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:43.612380 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-f3235684-091d-4e81-852a-6ab25cbab808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:43.694375 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3235684-091d-4e81-852a-6ab25cbab808 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:43.700200 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d87b20-468b-42b9-9858-39ad532ee809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:43.701001 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d87b20-468b-42b9-9858-39ad532ee809 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:43.701526 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d87b20-468b-42b9-9858-39ad532ee809 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:43.701777 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d87b20-468b-42b9-9858-39ad532ee809 admin admin] PATH_INFO: `/v3/users/6ace47df522b4cee9237d7c8f85a23ee` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:43.703053 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d87b20-468b-42b9-9858-39ad532ee809 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ace47df522b4cee9237d7c8f85a23ee)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:43.705039 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d87b20-468b-42b9-9858-39ad532ee809 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:43.762384 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-07d87b20-468b-42b9-9858-39ad532ee809 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ace47df522b4cee9237d7c8f85a23ee'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:43.779081 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-07d87b20-468b-42b9-9858-39ad532ee809 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ace47df522b4cee9237d7c8f85a23ee'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:43.786858 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-07d87b20-468b-42b9-9858-39ad532ee809 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ace47df522b4cee9237d7c8f85a23ee'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:43.787609 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-07d87b20-468b-42b9-9858-39ad532ee809 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ace47df522b4cee9237d7c8f85a23ee'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:43.789619 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:INSERT=1,keystone:SELECT=90,keystone:DELETE=9 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:43.797222 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-07d87b20-468b-42b9-9858-39ad532ee809 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ace47df522b4cee9237d7c8f85a23ee'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:43.809551 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1040/2094] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:14:43 2022] DELETE /identity/v3/users/6ace47df522b4cee9237d7c8f85a23ee => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:43.815116 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc98e5-9f80-42df-94a6-71e65f0aa39a None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:43.815635 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc98e5-9f80-42df-94a6-71e65f0aa39a None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:43.815988 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc98e5-9f80-42df-94a6-71e65f0aa39a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:43.851166 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-14bc98e5-9f80-42df-94a6-71e65f0aa39a None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:43.853456 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-14bc98e5-9f80-42df-94a6-71e65f0aa39a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:43.966051 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1055/2095] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:43.977692 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d87b20-468b-42b9-9858-39ad532ee809 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:43.978401 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-07d87b20-468b-42b9-9858-39ad532ee809 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:44.049361 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d87b20-468b-42b9-9858-39ad532ee809 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:44.053140 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b4214-2504-445f-b22f-55e840d79a78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:44.053880 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b4214-2504-445f-b22f-55e840d79a78 service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:44.054163 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b4214-2504-445f-b22f-55e840d79a78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:44.054441 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b4214-2504-445f-b22f-55e840d79a78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:44.066045 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8b4214-2504-445f-b22f-55e840d79a78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:44.067633 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8b4214-2504-445f-b22f-55e840d79a78 service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:44.078386 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1041/2096] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:14:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:14:44.375095 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bc98e5-9f80-42df-94a6-71e65f0aa39a None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:44.376555 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-14bc98e5-9f80-42df-94a6-71e65f0aa39a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:44.391168 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bc98e5-9f80-42df-94a6-71e65f0aa39a None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:44.396213 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b037ecc0-6404-4d17-b39d-d7a58ff11809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:44.396866 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037ecc0-6404-4d17-b39d-d7a58ff11809 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:44.397164 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037ecc0-6404-4d17-b39d-d7a58ff11809 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:44.397529 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037ecc0-6404-4d17-b39d-d7a58ff11809 admin admin] PATH_INFO: `/v3/projects/c1f1fc48e15146cd8232dd039fea0b1f` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:44.398529 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b037ecc0-6404-4d17-b39d-d7a58ff11809 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1f1fc48e15146cd8232dd039fea0b1f)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:44.400491 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b037ecc0-6404-4d17-b39d-d7a58ff11809 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:44.432717 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:44.453266 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-b037ecc0-6404-4d17-b39d-d7a58ff11809 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1f1fc48e15146cd8232dd039fea0b1f', 'request_id': 'req-b037ecc0-6404-4d17-b39d-d7a58ff11809'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:44.461406 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-b037ecc0-6404-4d17-b39d-d7a58ff11809 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1f1fc48e15146cd8232dd039fea0b1f', 'request_id': 'req-b037ecc0-6404-4d17-b39d-d7a58ff11809'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:44.469570 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-b037ecc0-6404-4d17-b39d-d7a58ff11809 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1f1fc48e15146cd8232dd039fea0b1f', 'request_id': 'req-b037ecc0-6404-4d17-b39d-d7a58ff11809'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:44.486436 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-b037ecc0-6404-4d17-b39d-d7a58ff11809 admin admin] The token cache is being invalidate because project c1f1fc48e15146cd8232dd039fea0b1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:14:44.486849 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-b037ecc0-6404-4d17-b39d-d7a58ff11809 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:44.487623 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-b037ecc0-6404-4d17-b39d-d7a58ff11809 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:44.489774 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1056/2097] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:14:44 2022] DELETE /identity/v3/projects/c1f1fc48e15146cd8232dd039fea0b1f => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:44.493535 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b4214-2504-445f-b22f-55e840d79a78 service neutron] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:44.505224 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b4214-2504-445f-b22f-55e840d79a78 service neutron] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:44.567935 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf1e7ea-4b58-4fb9-b1d8-f7441fec71f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:44.568611 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf1e7ea-4b58-4fb9-b1d8-f7441fec71f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:44.568890 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf1e7ea-4b58-4fb9-b1d8-f7441fec71f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:44.569162 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf1e7ea-4b58-4fb9-b1d8-f7441fec71f6 admin admin] PATH_INFO: `/v3/users/32704f34831a43d494174a045e938ab3` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:44.570296 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf1e7ea-4b58-4fb9-b1d8-f7441fec71f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32704f34831a43d494174a045e938ab3)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:44.572298 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf1e7ea-4b58-4fb9-b1d8-f7441fec71f6 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:44.617727 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-9cf1e7ea-4b58-4fb9-b1d8-f7441fec71f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32704f34831a43d494174a045e938ab3'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:44.630574 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-9cf1e7ea-4b58-4fb9-b1d8-f7441fec71f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32704f34831a43d494174a045e938ab3'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:44.637840 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-9cf1e7ea-4b58-4fb9-b1d8-f7441fec71f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32704f34831a43d494174a045e938ab3'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:44.638547 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-9cf1e7ea-4b58-4fb9-b1d8-f7441fec71f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32704f34831a43d494174a045e938ab3'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:44.647365 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-9cf1e7ea-4b58-4fb9-b1d8-f7441fec71f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32704f34831a43d494174a045e938ab3'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:44.653319 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1042/2098] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:14:44 2022] DELETE /identity/v3/users/32704f34831a43d494174a045e938ab3 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:44.921448 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b037ecc0-6404-4d17-b39d-d7a58ff11809 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:44.922978 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-b037ecc0-6404-4d17-b39d-d7a58ff11809 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:44.937098 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b037ecc0-6404-4d17-b39d-d7a58ff11809 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:45.004348 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a01608-cac8-47a6-9220-2c3714735438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:45.004924 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a01608-cac8-47a6-9220-2c3714735438 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:45.005207 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a01608-cac8-47a6-9220-2c3714735438 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:45.005501 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a01608-cac8-47a6-9220-2c3714735438 admin admin] PATH_INFO: `/v3/projects/fe955ea6c89a4414a32e35cc295ae303` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:45.006625 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a01608-cac8-47a6-9220-2c3714735438 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe955ea6c89a4414a32e35cc295ae303)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:45.008619 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a01608-cac8-47a6-9220-2c3714735438 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:45.059812 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-11a01608-cac8-47a6-9220-2c3714735438 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe955ea6c89a4414a32e35cc295ae303', 'request_id': 'req-11a01608-cac8-47a6-9220-2c3714735438'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:45.071158 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-11a01608-cac8-47a6-9220-2c3714735438 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe955ea6c89a4414a32e35cc295ae303', 'request_id': 'req-11a01608-cac8-47a6-9220-2c3714735438'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:45.077594 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-11a01608-cac8-47a6-9220-2c3714735438 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe955ea6c89a4414a32e35cc295ae303', 'request_id': 'req-11a01608-cac8-47a6-9220-2c3714735438'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:45.090472 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-11a01608-cac8-47a6-9220-2c3714735438 admin admin] The token cache is being invalidate because project fe955ea6c89a4414a32e35cc295ae303 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:14:45.090761 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-11a01608-cac8-47a6-9220-2c3714735438 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:45.091410 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-11a01608-cac8-47a6-9220-2c3714735438 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:45.093571 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1057/2099] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:14:44 2022] DELETE /identity/v3/projects/fe955ea6c89a4414a32e35cc295ae303 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:14:45.097623 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf1e7ea-4b58-4fb9-b1d8-f7441fec71f6 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:45.099024 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-9cf1e7ea-4b58-4fb9-b1d8-f7441fec71f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:45.112044 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf1e7ea-4b58-4fb9-b1d8-f7441fec71f6 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:45.120797 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:45.170205 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa31fca-de6c-4dcd-bb3c-52c49fb4ab99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:45.170867 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa31fca-de6c-4dcd-bb3c-52c49fb4ab99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:45.171127 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa31fca-de6c-4dcd-bb3c-52c49fb4ab99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:45.171434 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa31fca-de6c-4dcd-bb3c-52c49fb4ab99 admin admin] PATH_INFO: `/v3/users/3b18afe1160f417e8ca28f4a5fb7b9b5` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:45.172626 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa31fca-de6c-4dcd-bb3c-52c49fb4ab99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b18afe1160f417e8ca28f4a5fb7b9b5)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:45.174577 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa31fca-de6c-4dcd-bb3c-52c49fb4ab99 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:45.220554 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-9fa31fca-de6c-4dcd-bb3c-52c49fb4ab99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b18afe1160f417e8ca28f4a5fb7b9b5'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:45.234118 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-9fa31fca-de6c-4dcd-bb3c-52c49fb4ab99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b18afe1160f417e8ca28f4a5fb7b9b5'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:45.241857 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-9fa31fca-de6c-4dcd-bb3c-52c49fb4ab99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b18afe1160f417e8ca28f4a5fb7b9b5'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:45.242582 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-9fa31fca-de6c-4dcd-bb3c-52c49fb4ab99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b18afe1160f417e8ca28f4a5fb7b9b5'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:45.250017 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-9fa31fca-de6c-4dcd-bb3c-52c49fb4ab99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b18afe1160f417e8ca28f4a5fb7b9b5'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:45.256056 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1043/2100] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:14:45 2022] DELETE /identity/v3/users/3b18afe1160f417e8ca28f4a5fb7b9b5 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:14:45.536200 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a01608-cac8-47a6-9220-2c3714735438 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:45.537770 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-11a01608-cac8-47a6-9220-2c3714735438 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:45.563766 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a01608-cac8-47a6-9220-2c3714735438 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:45.611110 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:45.627753 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f5adf8-df86-48c2-9057-9c568b6a6f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:45.628435 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f5adf8-df86-48c2-9057-9c568b6a6f9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:45.628815 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f5adf8-df86-48c2-9057-9c568b6a6f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:45.629133 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f5adf8-df86-48c2-9057-9c568b6a6f9d admin admin] PATH_INFO: `/v3/projects/d5321991bcbb442bb757cddf7a2914f1` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:45.630317 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f5adf8-df86-48c2-9057-9c568b6a6f9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5321991bcbb442bb757cddf7a2914f1)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:45.632358 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f5adf8-df86-48c2-9057-9c568b6a6f9d admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:45.685368 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-92f5adf8-df86-48c2-9057-9c568b6a6f9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5321991bcbb442bb757cddf7a2914f1', 'request_id': 'req-92f5adf8-df86-48c2-9057-9c568b6a6f9d'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:45.692490 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-92f5adf8-df86-48c2-9057-9c568b6a6f9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5321991bcbb442bb757cddf7a2914f1', 'request_id': 'req-92f5adf8-df86-48c2-9057-9c568b6a6f9d'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:45.698196 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-92f5adf8-df86-48c2-9057-9c568b6a6f9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5321991bcbb442bb757cddf7a2914f1', 'request_id': 'req-92f5adf8-df86-48c2-9057-9c568b6a6f9d'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:45.713865 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-92f5adf8-df86-48c2-9057-9c568b6a6f9d admin admin] The token cache is being invalidate because project d5321991bcbb442bb757cddf7a2914f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:14:45.714157 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-92f5adf8-df86-48c2-9057-9c568b6a6f9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:45.714823 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-92f5adf8-df86-48c2-9057-9c568b6a6f9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:45.716360 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1058/2101] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:14:45 2022] DELETE /identity/v3/projects/d5321991bcbb442bb757cddf7a2914f1 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:14:45.721522 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c362d563-590e-4935-b959-5fa68563e62b None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:45.721972 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c362d563-590e-4935-b959-5fa68563e62b None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:45.722060 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c362d563-590e-4935-b959-5fa68563e62b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:45.742787 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-c362d563-590e-4935-b959-5fa68563e62b None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:45.743989 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-c362d563-590e-4935-b959-5fa68563e62b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:45.805640 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1044/2102] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:45.814290 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b28ed-5687-418a-90e4-c298a2995e80 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:45.814574 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b28ed-5687-418a-90e4-c298a2995e80 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:45.814881 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b28ed-5687-418a-90e4-c298a2995e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:45.835328 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-c34b28ed-5687-418a-90e4-c298a2995e80 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:45.836652 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-c34b28ed-5687-418a-90e4-c298a2995e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:45.845578 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1059/2103] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:45.849940 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c362d563-590e-4935-b959-5fa68563e62b None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:45.860459 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c362d563-590e-4935-b959-5fa68563e62b None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:45.865141 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af70442-c651-450d-aa05-58d349bcd610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:45.865735 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af70442-c651-450d-aa05-58d349bcd610 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:45.866009 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af70442-c651-450d-aa05-58d349bcd610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:45.866278 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af70442-c651-450d-aa05-58d349bcd610 admin admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:45.866926 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af70442-c651-450d-aa05-58d349bcd610 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:14:45.867227 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af70442-c651-450d-aa05-58d349bcd610 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:45.868871 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af70442-c651-450d-aa05-58d349bcd610 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:45.880894 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1045/2104] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:14:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:14:45.885382 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34b28ed-5687-418a-90e4-c298a2995e80 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:45.894794 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34b28ed-5687-418a-90e4-c298a2995e80 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:45.899532 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1422d2e-7341-4c25-875e-64f8f629d29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:45.900125 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1422d2e-7341-4c25-875e-64f8f629d29f admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:45.900432 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1422d2e-7341-4c25-875e-64f8f629d29f admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:45.900705 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1422d2e-7341-4c25-875e-64f8f629d29f admin admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:45.901343 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1422d2e-7341-4c25-875e-64f8f629d29f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:45.903237 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1422d2e-7341-4c25-875e-64f8f629d29f admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:45.935591 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1060/2105] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:14:45 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:14:45.941643 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af70442-c651-450d-aa05-58d349bcd610 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:45.951813 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af70442-c651-450d-aa05-58d349bcd610 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:45.968610 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:46.013356 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d251c-6ad3-4d1c-99ad-582bd848a96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:46.014018 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d251c-6ad3-4d1c-99ad-582bd848a96c admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:46.014296 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d251c-6ad3-4d1c-99ad-582bd848a96c admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:46.014568 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d251c-6ad3-4d1c-99ad-582bd848a96c admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:46.015193 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45d251c-6ad3-4d1c-99ad-582bd848a96c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:46.017158 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45d251c-6ad3-4d1c-99ad-582bd848a96c admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:46.039307 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1046/2106] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:14:45 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:14:46.043479 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1422d2e-7341-4c25-875e-64f8f629d29f admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:46.054558 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1422d2e-7341-4c25-875e-64f8f629d29f admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:46.059541 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae0d08f-6ecb-494a-b724-37af705a593d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:46.060309 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae0d08f-6ecb-494a-b724-37af705a593d admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:46.061208 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae0d08f-6ecb-494a-b724-37af705a593d admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:46.061855 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae0d08f-6ecb-494a-b724-37af705a593d admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:46.062614 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae0d08f-6ecb-494a-b724-37af705a593d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:46.064444 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae0d08f-6ecb-494a-b724-37af705a593d admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:46.074983 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1061/2107] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:14:46.079367 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d251c-6ad3-4d1c-99ad-582bd848a96c admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:46.089655 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d251c-6ad3-4d1c-99ad-582bd848a96c admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:46.094538 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff0027-e288-41bd-9a44-af8bd9d14096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:46.095192 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff0027-e288-41bd-9a44-af8bd9d14096 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:46.095476 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff0027-e288-41bd-9a44-af8bd9d14096 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:46.095752 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff0027-e288-41bd-9a44-af8bd9d14096 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:46.096427 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edff0027-e288-41bd-9a44-af8bd9d14096 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:46.098109 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edff0027-e288-41bd-9a44-af8bd9d14096 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:46.109340 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1047/2108] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:14:46.113914 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae0d08f-6ecb-494a-b724-37af705a593d admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:46.125055 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae0d08f-6ecb-494a-b724-37af705a593d admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:46.129964 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd82f1c-b377-4df6-9eba-1da5875e5c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:46.130962 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd82f1c-b377-4df6-9eba-1da5875e5c7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:46.131241 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd82f1c-b377-4df6-9eba-1da5875e5c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:46.131512 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd82f1c-b377-4df6-9eba-1da5875e5c7c admin admin] PATH_INFO: `/v3/projects/6964bb361f3b4aa68ad90dfd79bf6fde/users/78b7aef9439549aa9767d4397a9f9c6d/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:46.148714 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd82f1c-b377-4df6-9eba-1da5875e5c7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6964bb361f3b4aa68ad90dfd79bf6fde, user_id=78b7aef9439549aa9767d4397a9f9c6d, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:46.151394 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd82f1c-b377-4df6-9eba-1da5875e5c7c admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:46.166394 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1062/2109] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:14:46 2022] PUT /identity/v3/projects/6964bb361f3b4aa68ad90dfd79bf6fde/users/78b7aef9439549aa9767d4397a9f9c6d/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:14:46.177230 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8651de-1ba5-4908-aac0-5e6a42e7e672 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:46.177510 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8651de-1ba5-4908-aac0-5e6a42e7e672 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:46.177789 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8651de-1ba5-4908-aac0-5e6a42e7e672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:46.206396 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-de8651de-1ba5-4908-aac0-5e6a42e7e672 None None] MFA Rules not processed for user `78b7aef9439549aa9767d4397a9f9c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:46.207633 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-de8651de-1ba5-4908-aac0-5e6a42e7e672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:46.278639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1048/2110] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:46.291576 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd82f1c-b377-4df6-9eba-1da5875e5c7c admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:46.293327 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-9dd82f1c-b377-4df6-9eba-1da5875e5c7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:46.353053 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd82f1c-b377-4df6-9eba-1da5875e5c7c admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:46.356652 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7340a38c-0c9c-4829-aaf1-15eae24a5cd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:46.357280 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7340a38c-0c9c-4829-aaf1-15eae24a5cd1 service cinder] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:46.358255 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7340a38c-0c9c-4829-aaf1-15eae24a5cd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:46.358539 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7340a38c-0c9c-4829-aaf1-15eae24a5cd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:46.368588 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7340a38c-0c9c-4829-aaf1-15eae24a5cd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:46.370291 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7340a38c-0c9c-4829-aaf1-15eae24a5cd1 service cinder] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:46.380277 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1063/2111] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:14:46 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:14:48.488505 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8651de-1ba5-4908-aac0-5e6a42e7e672 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:48.490044 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-de8651de-1ba5-4908-aac0-5e6a42e7e672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:48.562781 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8651de-1ba5-4908-aac0-5e6a42e7e672 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:48.567769 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa0d5f7-232c-452d-a42d-af0223cfced1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:48.568470 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa0d5f7-232c-452d-a42d-af0223cfced1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:48.568818 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa0d5f7-232c-452d-a42d-af0223cfced1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:48.569282 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa0d5f7-232c-452d-a42d-af0223cfced1 admin admin] PATH_INFO: `/v3/users/e20e42b4418f41609cd8c3341418e56f` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:48.570471 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa0d5f7-232c-452d-a42d-af0223cfced1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e20e42b4418f41609cd8c3341418e56f)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:48.572456 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa0d5f7-232c-452d-a42d-af0223cfced1 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:48.611378 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:48.633081 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-cfa0d5f7-232c-452d-a42d-af0223cfced1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e20e42b4418f41609cd8c3341418e56f'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:48.646436 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-cfa0d5f7-232c-452d-a42d-af0223cfced1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e20e42b4418f41609cd8c3341418e56f'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:48.653197 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-cfa0d5f7-232c-452d-a42d-af0223cfced1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e20e42b4418f41609cd8c3341418e56f'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:48.654017 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-cfa0d5f7-232c-452d-a42d-af0223cfced1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e20e42b4418f41609cd8c3341418e56f'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:48.662715 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-cfa0d5f7-232c-452d-a42d-af0223cfced1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e20e42b4418f41609cd8c3341418e56f'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:48.669098 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1049/2112] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:14:48 2022] DELETE /identity/v3/users/e20e42b4418f41609cd8c3341418e56f => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:48.674394 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e02ed3-c6c2-4ce0-999f-fb55e57d20c6 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:48.674864 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e02ed3-c6c2-4ce0-999f-fb55e57d20c6 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:48.674958 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e02ed3-c6c2-4ce0-999f-fb55e57d20c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:48.697984 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-62e02ed3-c6c2-4ce0-999f-fb55e57d20c6 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:48.699141 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-62e02ed3-c6c2-4ce0-999f-fb55e57d20c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:48.709345 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:48.768916 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1064/2113] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:48.778853 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa0d5f7-232c-452d-a42d-af0223cfced1 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:48.792340 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-cfa0d5f7-232c-452d-a42d-af0223cfced1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:48.842831 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa0d5f7-232c-452d-a42d-af0223cfced1 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:48.847369 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89c2f89-1fc9-40e4-b9c7-e6de5bfd5804 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:48.847991 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c2f89-1fc9-40e4-b9c7-e6de5bfd5804 service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:48.848456 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c2f89-1fc9-40e4-b9c7-e6de5bfd5804 service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:48.848735 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c2f89-1fc9-40e4-b9c7-e6de5bfd5804 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:48.858674 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89c2f89-1fc9-40e4-b9c7-e6de5bfd5804 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:48.860187 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89c2f89-1fc9-40e4-b9c7-e6de5bfd5804 service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:48.871725 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1050/2114] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:14:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:14:49.173632 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e02ed3-c6c2-4ce0-999f-fb55e57d20c6 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:49.175098 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-62e02ed3-c6c2-4ce0-999f-fb55e57d20c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:49.188074 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e02ed3-c6c2-4ce0-999f-fb55e57d20c6 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:49.193364 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c7e9b7-4774-4e89-89e9-9425011aa5e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:49.194013 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7e9b7-4774-4e89-89e9-9425011aa5e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:49.194302 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7e9b7-4774-4e89-89e9-9425011aa5e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:49.194566 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7e9b7-4774-4e89-89e9-9425011aa5e5 admin admin] PATH_INFO: `/v3/projects/322ea50d34cd4311812b0a75d89df303` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:49.195845 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c7e9b7-4774-4e89-89e9-9425011aa5e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=322ea50d34cd4311812b0a75d89df303)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:49.197815 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c7e9b7-4774-4e89-89e9-9425011aa5e5 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:49.271985 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-b4c7e9b7-4774-4e89-89e9-9425011aa5e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '322ea50d34cd4311812b0a75d89df303', 'request_id': 'req-b4c7e9b7-4774-4e89-89e9-9425011aa5e5'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:49.282404 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-b4c7e9b7-4774-4e89-89e9-9425011aa5e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '322ea50d34cd4311812b0a75d89df303', 'request_id': 'req-b4c7e9b7-4774-4e89-89e9-9425011aa5e5'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:49.297523 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-b4c7e9b7-4774-4e89-89e9-9425011aa5e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '322ea50d34cd4311812b0a75d89df303', 'request_id': 'req-b4c7e9b7-4774-4e89-89e9-9425011aa5e5'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:49.314617 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-b4c7e9b7-4774-4e89-89e9-9425011aa5e5 admin admin] The token cache is being invalidate because project 322ea50d34cd4311812b0a75d89df303 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:14:49.314912 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-b4c7e9b7-4774-4e89-89e9-9425011aa5e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:49.315685 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-b4c7e9b7-4774-4e89-89e9-9425011aa5e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:49.317410 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1065/2115] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:14:49 2022] DELETE /identity/v3/projects/322ea50d34cd4311812b0a75d89df303 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:49.322730 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fef90e-03f3-4d15-9e80-564829ff4b8b None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:49.323054 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fef90e-03f3-4d15-9e80-564829ff4b8b None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:49.323284 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fef90e-03f3-4d15-9e80-564829ff4b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:49.349919 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-30fef90e-03f3-4d15-9e80-564829ff4b8b None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:49.351166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-30fef90e-03f3-4d15-9e80-564829ff4b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:49.431597 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1051/2116] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:49.440658 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0e91e-759a-474e-a9b8-7099be14fbac None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:49.440944 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0e91e-759a-474e-a9b8-7099be14fbac None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:49.441209 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0e91e-759a-474e-a9b8-7099be14fbac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:49.461969 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-afe0e91e-759a-474e-a9b8-7099be14fbac None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:49.463201 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-afe0e91e-759a-474e-a9b8-7099be14fbac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:49.471294 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1066/2117] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:49.475646 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fef90e-03f3-4d15-9e80-564829ff4b8b None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:49.485965 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fef90e-03f3-4d15-9e80-564829ff4b8b None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:49.491775 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e833e4-f6a1-4173-b3b3-71e851c93e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:49.492421 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e833e4-f6a1-4173-b3b3-71e851c93e1d admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:49.492877 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e833e4-f6a1-4173-b3b3-71e851c93e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:49.493154 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e833e4-f6a1-4173-b3b3-71e851c93e1d admin admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:49.493819 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e833e4-f6a1-4173-b3b3-71e851c93e1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:14:49.494131 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e833e4-f6a1-4173-b3b3-71e851c93e1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:49.495769 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e833e4-f6a1-4173-b3b3-71e851c93e1d admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:49.520398 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1052/2118] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:14:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:14:49.522197 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe0e91e-759a-474e-a9b8-7099be14fbac None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:49.534945 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe0e91e-759a-474e-a9b8-7099be14fbac None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:49.542923 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e1e29-6c17-4d58-a96a-53e53c68af71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:49.543677 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e1e29-6c17-4d58-a96a-53e53c68af71 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:49.543957 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e1e29-6c17-4d58-a96a-53e53c68af71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:49.544269 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e1e29-6c17-4d58-a96a-53e53c68af71 admin admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:49.545070 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067e1e29-6c17-4d58-a96a-53e53c68af71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:49.547108 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067e1e29-6c17-4d58-a96a-53e53c68af71 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:49.575324 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1067/2119] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:14:49 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:14:49.580215 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e833e4-f6a1-4173-b3b3-71e851c93e1d admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:49.590181 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e833e4-f6a1-4173-b3b3-71e851c93e1d admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:49.630059 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:49.659121 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc4fead-2f0b-4222-89ab-66ac9bd7f848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:49.659767 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc4fead-2f0b-4222-89ab-66ac9bd7f848 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:49.660047 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc4fead-2f0b-4222-89ab-66ac9bd7f848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:49.660368 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc4fead-2f0b-4222-89ab-66ac9bd7f848 admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:49.661135 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc4fead-2f0b-4222-89ab-66ac9bd7f848 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:49.663385 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc4fead-2f0b-4222-89ab-66ac9bd7f848 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:49.687441 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1053/2120] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:14:49 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:14:49.691805 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e1e29-6c17-4d58-a96a-53e53c68af71 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:49.705020 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e1e29-6c17-4d58-a96a-53e53c68af71 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:49.710297 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaf671a-18bd-4c2a-9480-c6ef244d8439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:49.710975 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaf671a-18bd-4c2a-9480-c6ef244d8439 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:49.711317 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaf671a-18bd-4c2a-9480-c6ef244d8439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:49.711676 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaf671a-18bd-4c2a-9480-c6ef244d8439 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:49.712549 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaf671a-18bd-4c2a-9480-c6ef244d8439 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:49.714611 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaf671a-18bd-4c2a-9480-c6ef244d8439 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:49.733349 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1068/2121] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:14:49.739469 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc4fead-2f0b-4222-89ab-66ac9bd7f848 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:49.751434 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc4fead-2f0b-4222-89ab-66ac9bd7f848 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:49.755811 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e3572a-1c90-4661-a9db-1f496083b353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:49.756460 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3572a-1c90-4661-a9db-1f496083b353 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:49.756745 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3572a-1c90-4661-a9db-1f496083b353 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:49.757043 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3572a-1c90-4661-a9db-1f496083b353 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:49.757693 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e3572a-1c90-4661-a9db-1f496083b353 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:49.759569 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e3572a-1c90-4661-a9db-1f496083b353 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:49.773308 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1054/2122] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:14:49.777145 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaf671a-18bd-4c2a-9480-c6ef244d8439 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:49.790211 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaf671a-18bd-4c2a-9480-c6ef244d8439 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:49.799646 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c7043-c108-4b30-812c-9ddb4ae0ff66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:49.800765 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c7043-c108-4b30-812c-9ddb4ae0ff66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:49.801243 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c7043-c108-4b30-812c-9ddb4ae0ff66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:49.801694 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c7043-c108-4b30-812c-9ddb4ae0ff66 admin admin] PATH_INFO: `/v3/projects/275f1c82d4f34cbfa81591e5f5a13f63/users/5411bb3267704943aea31c4bf1b4c9d1/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:49.819185 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9c7043-c108-4b30-812c-9ddb4ae0ff66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=275f1c82d4f34cbfa81591e5f5a13f63, user_id=5411bb3267704943aea31c4bf1b4c9d1, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:49.821921 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9c7043-c108-4b30-812c-9ddb4ae0ff66 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:49.836558 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1069/2123] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:14:49 2022] PUT /identity/v3/projects/275f1c82d4f34cbfa81591e5f5a13f63/users/5411bb3267704943aea31c4bf1b4c9d1/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:49.842213 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf09dc-e079-40f4-856d-acbdeb12a262 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:49.842532 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf09dc-e079-40f4-856d-acbdeb12a262 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:49.842807 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf09dc-e079-40f4-856d-acbdeb12a262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:49.861926 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-cdcf09dc-e079-40f4-856d-acbdeb12a262 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:49.863175 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-cdcf09dc-e079-40f4-856d-acbdeb12a262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:49.922343 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1055/2124] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:49.932248 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c7043-c108-4b30-812c-9ddb4ae0ff66 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:49.934023 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-4a9c7043-c108-4b30-812c-9ddb4ae0ff66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:49.975209 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:49.994045 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c7043-c108-4b30-812c-9ddb4ae0ff66 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:49.998338 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a1d2d7-36c7-4b5f-ae80-89771d380e81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:49.999010 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a1d2d7-36c7-4b5f-ae80-89771d380e81 service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:49.999314 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a1d2d7-36c7-4b5f-ae80-89771d380e81 service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:49.999567 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a1d2d7-36c7-4b5f-ae80-89771d380e81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:50.008674 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a1d2d7-36c7-4b5f-ae80-89771d380e81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:50.010609 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a1d2d7-36c7-4b5f-ae80-89771d380e81 service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:50.021285 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1070/2125] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:14:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:14:50.029808 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcf09dc-e079-40f4-856d-acbdeb12a262 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:50.031408 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-cdcf09dc-e079-40f4-856d-acbdeb12a262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:50.043591 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcf09dc-e079-40f4-856d-acbdeb12a262 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:50.048736 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b666a7d5-3ff6-4ee2-a05d-9a0de9634db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:50.049379 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b666a7d5-3ff6-4ee2-a05d-9a0de9634db3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:50.049674 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b666a7d5-3ff6-4ee2-a05d-9a0de9634db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:50.049977 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b666a7d5-3ff6-4ee2-a05d-9a0de9634db3 admin admin] PATH_INFO: `/v3/users/78b7aef9439549aa9767d4397a9f9c6d` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:50.051167 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b666a7d5-3ff6-4ee2-a05d-9a0de9634db3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78b7aef9439549aa9767d4397a9f9c6d)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:50.053275 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b666a7d5-3ff6-4ee2-a05d-9a0de9634db3 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:50.101716 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-b666a7d5-3ff6-4ee2-a05d-9a0de9634db3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78b7aef9439549aa9767d4397a9f9c6d'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:50.114815 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-b666a7d5-3ff6-4ee2-a05d-9a0de9634db3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78b7aef9439549aa9767d4397a9f9c6d'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:50.122331 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-b666a7d5-3ff6-4ee2-a05d-9a0de9634db3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78b7aef9439549aa9767d4397a9f9c6d'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:50.123574 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-b666a7d5-3ff6-4ee2-a05d-9a0de9634db3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78b7aef9439549aa9767d4397a9f9c6d'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:50.131943 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-b666a7d5-3ff6-4ee2-a05d-9a0de9634db3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78b7aef9439549aa9767d4397a9f9c6d'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:50.138314 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1056/2126] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:14:50 2022] DELETE /identity/v3/users/78b7aef9439549aa9767d4397a9f9c6d => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:50.144026 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45467070-4087-41ca-aa56-426667f60585 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:50.144385 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45467070-4087-41ca-aa56-426667f60585 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:50.144691 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45467070-4087-41ca-aa56-426667f60585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:50.164383 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-45467070-4087-41ca-aa56-426667f60585 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:50.165605 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-45467070-4087-41ca-aa56-426667f60585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:50.231304 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1071/2127] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:50.241617 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b666a7d5-3ff6-4ee2-a05d-9a0de9634db3 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:50.243093 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-b666a7d5-3ff6-4ee2-a05d-9a0de9634db3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:50.278922 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:50.308887 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b666a7d5-3ff6-4ee2-a05d-9a0de9634db3 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:50.312603 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9f1c23-09ad-4ac6-940c-93eaca719917 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:50.313268 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9f1c23-09ad-4ac6-940c-93eaca719917 service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:50.313513 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9f1c23-09ad-4ac6-940c-93eaca719917 service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:50.314277 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9f1c23-09ad-4ac6-940c-93eaca719917 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:50.325757 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9f1c23-09ad-4ac6-940c-93eaca719917 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:50.327997 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9f1c23-09ad-4ac6-940c-93eaca719917 service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:50.343906 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1057/2128] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:14:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:14:50.989678 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45467070-4087-41ca-aa56-426667f60585 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:50.991416 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-45467070-4087-41ca-aa56-426667f60585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:51.015298 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45467070-4087-41ca-aa56-426667f60585 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:51.026291 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef85214d-6fa9-4e24-a669-3ab5a2255c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:51.027239 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef85214d-6fa9-4e24-a669-3ab5a2255c9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:51.028167 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef85214d-6fa9-4e24-a669-3ab5a2255c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:51.028623 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef85214d-6fa9-4e24-a669-3ab5a2255c9b admin admin] PATH_INFO: `/v3/projects/6964bb361f3b4aa68ad90dfd79bf6fde` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:51.031225 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef85214d-6fa9-4e24-a669-3ab5a2255c9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6964bb361f3b4aa68ad90dfd79bf6fde)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:51.037488 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef85214d-6fa9-4e24-a669-3ab5a2255c9b admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:51.119524 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-ef85214d-6fa9-4e24-a669-3ab5a2255c9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6964bb361f3b4aa68ad90dfd79bf6fde', 'request_id': 'req-ef85214d-6fa9-4e24-a669-3ab5a2255c9b'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:51.127024 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-ef85214d-6fa9-4e24-a669-3ab5a2255c9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6964bb361f3b4aa68ad90dfd79bf6fde', 'request_id': 'req-ef85214d-6fa9-4e24-a669-3ab5a2255c9b'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:51.134646 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-ef85214d-6fa9-4e24-a669-3ab5a2255c9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6964bb361f3b4aa68ad90dfd79bf6fde', 'request_id': 'req-ef85214d-6fa9-4e24-a669-3ab5a2255c9b'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:51.154389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-ef85214d-6fa9-4e24-a669-3ab5a2255c9b admin admin] The token cache is being invalidate because project 6964bb361f3b4aa68ad90dfd79bf6fde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:14:51.154723 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-ef85214d-6fa9-4e24-a669-3ab5a2255c9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:51.155424 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-ef85214d-6fa9-4e24-a669-3ab5a2255c9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:51.168900 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1072/2129] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:14:50 2022] DELETE /identity/v3/projects/6964bb361f3b4aa68ad90dfd79bf6fde => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:55.386837 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd46272-8a25-49fe-80b5-effd33f112dc None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:55.387326 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd46272-8a25-49fe-80b5-effd33f112dc None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:55.387402 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd46272-8a25-49fe-80b5-effd33f112dc None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:55.391046 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1058/2130] 10.222.0.42 () {54 vars in 921 bytes} [Thu Sep 8 01:14:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:14:55.490267 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239977a-4d59-408f-b6ee-92e55d1115c7 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:55.491097 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239977a-4d59-408f-b6ee-92e55d1115c7 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:55.491536 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239977a-4d59-408f-b6ee-92e55d1115c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:55.549226 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-8239977a-4d59-408f-b6ee-92e55d1115c7 None None] MFA Rules not processed for user `301d30bbd3de4cc69e96bb12729442c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:55.549954 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-8239977a-4d59-408f-b6ee-92e55d1115c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:55.629326 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd46272-8a25-49fe-80b5-effd33f112dc None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:55.640760 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-9dd46272-8a25-49fe-80b5-effd33f112dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:55.703850 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:55.745888 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1073/2131] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Sep 8 01:14:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:55.753351 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239977a-4d59-408f-b6ee-92e55d1115c7 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:55.755221 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-8239977a-4d59-408f-b6ee-92e55d1115c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:55.781892 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239977a-4d59-408f-b6ee-92e55d1115c7 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:55.798468 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb810d38-b83f-4c09-a782-833900117a9f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '301d30bbd3de4cc69e96bb12729442c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:55.799283 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb810d38-b83f-4c09-a782-833900117a9f service placement] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:55.799710 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb810d38-b83f-4c09-a782-833900117a9f service placement] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:55.800393 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb810d38-b83f-4c09-a782-833900117a9f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:55.838215 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb810d38-b83f-4c09-a782-833900117a9f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:55.839849 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb810d38-b83f-4c09-a782-833900117a9f service placement] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:55.849294 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd46272-8a25-49fe-80b5-effd33f112dc None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:55.856500 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a88d53c-1377-488e-a94f-ac8f22115cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:55.857205 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a88d53c-1377-488e-a94f-ac8f22115cc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:55.857486 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a88d53c-1377-488e-a94f-ac8f22115cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:55.857760 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a88d53c-1377-488e-a94f-ac8f22115cc8 admin admin] PATH_INFO: `/v3/users/36fe14a406444624a34a73e2ad60a75c` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:55.858952 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a88d53c-1377-488e-a94f-ac8f22115cc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36fe14a406444624a34a73e2ad60a75c)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:55.860988 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a88d53c-1377-488e-a94f-ac8f22115cc8 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:55.874785 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1074/2132] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:14:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:14:55.877160 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72313df-dbc6-4e46-812a-752b3dabcd6d None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:55.877451 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72313df-dbc6-4e46-812a-752b3dabcd6d None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:55.877731 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72313df-dbc6-4e46-812a-752b3dabcd6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:55.993281 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-2a88d53c-1377-488e-a94f-ac8f22115cc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36fe14a406444624a34a73e2ad60a75c'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:56.004932 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-b72313df-dbc6-4e46-812a-752b3dabcd6d None None] MFA Rules not processed for user `5411bb3267704943aea31c4bf1b4c9d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:56.006543 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-b72313df-dbc6-4e46-812a-752b3dabcd6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:56.026448 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-2a88d53c-1377-488e-a94f-ac8f22115cc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36fe14a406444624a34a73e2ad60a75c'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:56.035340 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-2a88d53c-1377-488e-a94f-ac8f22115cc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36fe14a406444624a34a73e2ad60a75c'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:56.036202 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-2a88d53c-1377-488e-a94f-ac8f22115cc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36fe14a406444624a34a73e2ad60a75c'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:56.062802 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-2a88d53c-1377-488e-a94f-ac8f22115cc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36fe14a406444624a34a73e2ad60a75c'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:56.070740 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1059/2133] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:14:55 2022] DELETE /identity/v3/users/36fe14a406444624a34a73e2ad60a75c => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:56.127135 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1075/2134] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:56.131805 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a88d53c-1377-488e-a94f-ac8f22115cc8 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:56.134563 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-2a88d53c-1377-488e-a94f-ac8f22115cc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:56.250605 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:56.266797 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a88d53c-1377-488e-a94f-ac8f22115cc8 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:56.276294 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bb2546-c315-404e-8c20-d0a9408ff1e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:56.277062 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb2546-c315-404e-8c20-d0a9408ff1e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:56.277449 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb2546-c315-404e-8c20-d0a9408ff1e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:56.277824 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb2546-c315-404e-8c20-d0a9408ff1e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:56.278662 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bb2546-c315-404e-8c20-d0a9408ff1e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:56.283494 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bb2546-c315-404e-8c20-d0a9408ff1e1 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:56.308190 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72313df-dbc6-4e46-812a-752b3dabcd6d None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:56.309712 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-b72313df-dbc6-4e46-812a-752b3dabcd6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:56.335371 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72313df-dbc6-4e46-812a-752b3dabcd6d None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:56.347845 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1060/2135] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:14:56 2022] POST /identity/v3/projects => generated 383 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:14:56.351390 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bb2546-c315-404e-8c20-d0a9408ff1e1 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:56.353479 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a2b740-8755-4673-b8b7-53cdb737f165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:56.354421 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a2b740-8755-4673-b8b7-53cdb737f165 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:56.355182 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a2b740-8755-4673-b8b7-53cdb737f165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:56.355476 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a2b740-8755-4673-b8b7-53cdb737f165 admin admin] PATH_INFO: `/v3/projects/c1186c0555f345b6915c81b1f33b84bc` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:56.356722 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a2b740-8755-4673-b8b7-53cdb737f165 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1186c0555f345b6915c81b1f33b84bc)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:56.359258 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a2b740-8755-4673-b8b7-53cdb737f165 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:56.426758 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-20a2b740-8755-4673-b8b7-53cdb737f165 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1186c0555f345b6915c81b1f33b84bc', 'request_id': 'req-20a2b740-8755-4673-b8b7-53cdb737f165'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:56.427268 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bb2546-c315-404e-8c20-d0a9408ff1e1 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:56.434230 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-20a2b740-8755-4673-b8b7-53cdb737f165 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1186c0555f345b6915c81b1f33b84bc', 'request_id': 'req-20a2b740-8755-4673-b8b7-53cdb737f165'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:56.441595 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-20a2b740-8755-4673-b8b7-53cdb737f165 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1186c0555f345b6915c81b1f33b84bc', 'request_id': 'req-20a2b740-8755-4673-b8b7-53cdb737f165'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:56.459577 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-20a2b740-8755-4673-b8b7-53cdb737f165 admin admin] The token cache is being invalidate because project c1186c0555f345b6915c81b1f33b84bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:14:56.459869 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-20a2b740-8755-4673-b8b7-53cdb737f165 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:56.460547 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-20a2b740-8755-4673-b8b7-53cdb737f165 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:56.462284 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1076/2136] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:14:56 2022] DELETE /identity/v3/projects/c1186c0555f345b6915c81b1f33b84bc => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:14:56.466947 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee381df3-5abc-479f-ae32-3f5f0e03876e None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:56.467229 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee381df3-5abc-479f-ae32-3f5f0e03876e None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:56.467702 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee381df3-5abc-479f-ae32-3f5f0e03876e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:56.488989 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-ee381df3-5abc-479f-ae32-3f5f0e03876e None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:56.490131 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-ee381df3-5abc-479f-ae32-3f5f0e03876e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:56.494421 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d202113-2d20-436b-9a24-47ea7d0163d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:56.495261 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d202113-2d20-436b-9a24-47ea7d0163d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:56.495557 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d202113-2d20-436b-9a24-47ea7d0163d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:56.495933 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d202113-2d20-436b-9a24-47ea7d0163d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:56.496706 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d202113-2d20-436b-9a24-47ea7d0163d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:56.499162 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1077/2137] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:14:56.499651 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d202113-2d20-436b-9a24-47ea7d0163d4 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:56.507356 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5865bf05-be05-42f2-9637-b45f0068bbb9 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:56.507865 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5865bf05-be05-42f2-9637-b45f0068bbb9 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:56.508199 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5865bf05-be05-42f2-9637-b45f0068bbb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:56.530022 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1061/2138] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:14:56 2022] POST /identity/v3/users => generated 298 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:14:56.530756 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-5865bf05-be05-42f2-9637-b45f0068bbb9 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:56.532338 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-5865bf05-be05-42f2-9637-b45f0068bbb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:56.534572 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d202113-2d20-436b-9a24-47ea7d0163d4 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:56.537333 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-4d202113-2d20-436b-9a24-47ea7d0163d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:56.541817 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1078/2139] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:14:56.545894 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5865bf05-be05-42f2-9637-b45f0068bbb9 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:56.553198 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d202113-2d20-436b-9a24-47ea7d0163d4 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:56.557411 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5865bf05-be05-42f2-9637-b45f0068bbb9 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:56.559565 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67873447-6a23-4b00-9051-9c21fc649839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:56.560381 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67873447-6a23-4b00-9051-9c21fc649839 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:56.560983 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67873447-6a23-4b00-9051-9c21fc649839 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:56.561098 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67873447-6a23-4b00-9051-9c21fc649839 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:56.562153 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67873447-6a23-4b00-9051-9c21fc649839 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:56.564605 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc0909a-e65a-4c6b-89f8-7df55756f401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:56.565301 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc0909a-e65a-4c6b-89f8-7df55756f401 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:56.565676 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc0909a-e65a-4c6b-89f8-7df55756f401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:56.565918 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc0909a-e65a-4c6b-89f8-7df55756f401 admin admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:56.566569 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc0909a-e65a-4c6b-89f8-7df55756f401 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:14:56.566874 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc0909a-e65a-4c6b-89f8-7df55756f401 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:56.568741 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc0909a-e65a-4c6b-89f8-7df55756f401 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:56.569447 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67873447-6a23-4b00-9051-9c21fc649839 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:56.583844 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1062/2140] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:14:56 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 01:14:56.585659 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1079/2141] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:14:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:14:56.588224 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67873447-6a23-4b00-9051-9c21fc649839 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:56.590433 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc0909a-e65a-4c6b-89f8-7df55756f401 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:56.602078 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67873447-6a23-4b00-9051-9c21fc649839 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:56.607225 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc0909a-e65a-4c6b-89f8-7df55756f401 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:56.607600 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f481e-1df3-4bc1-81d7-6cbdda5cfffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:56.608399 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f481e-1df3-4bc1-81d7-6cbdda5cfffc admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:56.608742 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f481e-1df3-4bc1-81d7-6cbdda5cfffc admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:56.608962 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f481e-1df3-4bc1-81d7-6cbdda5cfffc admin admin] PATH_INFO: `/v3/projects/07b8c4b08bb041169bb932e2de46711a/users/dc29e7a08e1d4f0fbd259ee49b4a97d0/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:56.613595 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee977f-a80e-4c76-b7ee-6fafbfc56d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:56.614211 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee977f-a80e-4c76-b7ee-6fafbfc56d1c admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:56.614479 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee977f-a80e-4c76-b7ee-6fafbfc56d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:56.614753 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee977f-a80e-4c76-b7ee-6fafbfc56d1c admin admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:56.615383 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee977f-a80e-4c76-b7ee-6fafbfc56d1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:56.617335 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee977f-a80e-4c76-b7ee-6fafbfc56d1c admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:56.631477 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194f481e-1df3-4bc1-81d7-6cbdda5cfffc admin admin] RBAC: Authorizing `identity:create_grant(project_id=07b8c4b08bb041169bb932e2de46711a, user_id=dc29e7a08e1d4f0fbd259ee49b4a97d0, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:56.633652 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194f481e-1df3-4bc1-81d7-6cbdda5cfffc admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:56.651455 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1080/2142] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:14:56 2022] POST /identity/v3/projects => generated 383 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 01:14:56.652243 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1063/2143] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:14:56 2022] PUT /identity/v3/projects/07b8c4b08bb041169bb932e2de46711a/users/dc29e7a08e1d4f0fbd259ee49b4a97d0/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:14:56.655879 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee977f-a80e-4c76-b7ee-6fafbfc56d1c admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:56.657185 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f481e-1df3-4bc1-81d7-6cbdda5cfffc admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:56.667798 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:56.715833 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:56.749674 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee977f-a80e-4c76-b7ee-6fafbfc56d1c admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:56.755481 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f481e-1df3-4bc1-81d7-6cbdda5cfffc admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:56.755658 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7445b21-a6ab-4d69-b115-72ef3e48ae29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:56.756341 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7445b21-a6ab-4d69-b115-72ef3e48ae29 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:56.756717 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7445b21-a6ab-4d69-b115-72ef3e48ae29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:56.756927 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7445b21-a6ab-4d69-b115-72ef3e48ae29 admin admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:56.757618 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7445b21-a6ab-4d69-b115-72ef3e48ae29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:56.760070 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7445b21-a6ab-4d69-b115-72ef3e48ae29 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:56.760875 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58277b1b-4541-4e74-bd7e-7ea75e908363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:56.761634 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58277b1b-4541-4e74-bd7e-7ea75e908363 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:56.761957 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58277b1b-4541-4e74-bd7e-7ea75e908363 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:56.762356 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58277b1b-4541-4e74-bd7e-7ea75e908363 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:56.763256 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58277b1b-4541-4e74-bd7e-7ea75e908363 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:56.765580 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58277b1b-4541-4e74-bd7e-7ea75e908363 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:56.778081 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1064/2144] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:14:56 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:14:56.782578 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58277b1b-4541-4e74-bd7e-7ea75e908363 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:56.792337 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1081/2145] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:14:56 2022] POST /identity/v3/users => generated 292 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:14:56.794109 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58277b1b-4541-4e74-bd7e-7ea75e908363 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:56.796398 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7445b21-a6ab-4d69-b115-72ef3e48ae29 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:56.799627 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6872adc7-3d17-445a-b009-89da05a26c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:56.800343 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872adc7-3d17-445a-b009-89da05a26c09 admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:56.800627 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872adc7-3d17-445a-b009-89da05a26c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:56.800907 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872adc7-3d17-445a-b009-89da05a26c09 admin admin] PATH_INFO: `/v3/projects/07b8c4b08bb041169bb932e2de46711a/users/dc29e7a08e1d4f0fbd259ee49b4a97d0/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:56.803106 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6872adc7-3d17-445a-b009-89da05a26c09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07b8c4b08bb041169bb932e2de46711a, user_id=dc29e7a08e1d4f0fbd259ee49b4a97d0, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:56.806029 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6872adc7-3d17-445a-b009-89da05a26c09 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:56.809094 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7445b21-a6ab-4d69-b115-72ef3e48ae29 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:56.815328 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01eb3d3-6e9f-4cdb-8b18-810eed1917b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:56.816047 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01eb3d3-6e9f-4cdb-8b18-810eed1917b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:56.816490 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01eb3d3-6e9f-4cdb-8b18-810eed1917b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:56.816858 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01eb3d3-6e9f-4cdb-8b18-810eed1917b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:56.817636 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01eb3d3-6e9f-4cdb-8b18-810eed1917b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:56.818613 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1065/2146] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:14:56 2022] PUT /identity/v3/projects/07b8c4b08bb041169bb932e2de46711a/users/dc29e7a08e1d4f0fbd259ee49b4a97d0/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:14:56.819718 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01eb3d3-6e9f-4cdb-8b18-810eed1917b9 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:56.836225 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1082/2147] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:14:56 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:14:56.840726 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6872adc7-3d17-445a-b009-89da05a26c09 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:56.942992 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6872adc7-3d17-445a-b009-89da05a26c09 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:56.948946 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a108b3c-0321-4f7b-8686-f2d82f765f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:56.949576 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a108b3c-0321-4f7b-8686-f2d82f765f0d admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:56.949859 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a108b3c-0321-4f7b-8686-f2d82f765f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:56.950134 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a108b3c-0321-4f7b-8686-f2d82f765f0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:56.951282 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a108b3c-0321-4f7b-8686-f2d82f765f0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:56.952840 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a108b3c-0321-4f7b-8686-f2d82f765f0d admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:56.964245 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1066/2148] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:14:56 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:14:56.969252 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01eb3d3-6e9f-4cdb-8b18-810eed1917b9 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:56.981609 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01eb3d3-6e9f-4cdb-8b18-810eed1917b9 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:56.987528 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4818646e-4777-41c0-a777-34122db67c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:56.988245 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4818646e-4777-41c0-a777-34122db67c4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:56.988520 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4818646e-4777-41c0-a777-34122db67c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:56.989054 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4818646e-4777-41c0-a777-34122db67c4c admin admin] PATH_INFO: `/v3/projects/e856bb81a717482cbea65029d521700d/users/83fea9762aaf4766a23f1774ff1411fe/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:57.012519 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4818646e-4777-41c0-a777-34122db67c4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e856bb81a717482cbea65029d521700d, user_id=83fea9762aaf4766a23f1774ff1411fe, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:57.016475 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4818646e-4777-41c0-a777-34122db67c4c admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:57.036560 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1083/2149] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:14:56 2022] PUT /identity/v3/projects/e856bb81a717482cbea65029d521700d/users/83fea9762aaf4766a23f1774ff1411fe/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:14:57.038710 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eddcd7-5a6a-48f2-94a2-3324d24a84a8 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:57.039057 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eddcd7-5a6a-48f2-94a2-3324d24a84a8 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:57.039398 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eddcd7-5a6a-48f2-94a2-3324d24a84a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:57.062280 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-f6eddcd7-5a6a-48f2-94a2-3324d24a84a8 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:57.063599 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-f6eddcd7-5a6a-48f2-94a2-3324d24a84a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:57.153441 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1067/2150] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:57.154735 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4818646e-4777-41c0-a777-34122db67c4c admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:57.157169 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-4818646e-4777-41c0-a777-34122db67c4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:57.236335 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4818646e-4777-41c0-a777-34122db67c4c admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:57.242462 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5344039-5636-44bb-b51b-9e0d8e64f434 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:57.243149 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5344039-5636-44bb-b51b-9e0d8e64f434 service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:57.243507 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5344039-5636-44bb-b51b-9e0d8e64f434 service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:57.243842 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5344039-5636-44bb-b51b-9e0d8e64f434 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:57.256565 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5344039-5636-44bb-b51b-9e0d8e64f434 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:57.258361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5344039-5636-44bb-b51b-9e0d8e64f434 service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:57.288283 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1084/2151] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:14:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:14:58.400805 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eddcd7-5a6a-48f2-94a2-3324d24a84a8 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:58.402303 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-f6eddcd7-5a6a-48f2-94a2-3324d24a84a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:58.422302 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eddcd7-5a6a-48f2-94a2-3324d24a84a8 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:58.439785 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9b0742-ca63-4cd7-8ced-9bf2d17716cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:58.440553 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b0742-ca63-4cd7-8ced-9bf2d17716cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:58.440843 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b0742-ca63-4cd7-8ced-9bf2d17716cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:58.441124 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b0742-ca63-4cd7-8ced-9bf2d17716cf admin admin] PATH_INFO: `/v3/users/3f8fd6b58b184bac824348b6a19dec4f` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:58.443234 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9b0742-ca63-4cd7-8ced-9bf2d17716cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f8fd6b58b184bac824348b6a19dec4f)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:58.445258 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9b0742-ca63-4cd7-8ced-9bf2d17716cf admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:58.548122 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-ab9b0742-ca63-4cd7-8ced-9bf2d17716cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f8fd6b58b184bac824348b6a19dec4f'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:58.555467 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:58.574709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-ab9b0742-ca63-4cd7-8ced-9bf2d17716cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f8fd6b58b184bac824348b6a19dec4f'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:58.584960 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-ab9b0742-ca63-4cd7-8ced-9bf2d17716cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f8fd6b58b184bac824348b6a19dec4f'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:58.585838 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-ab9b0742-ca63-4cd7-8ced-9bf2d17716cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f8fd6b58b184bac824348b6a19dec4f'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:58.597292 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-ab9b0742-ca63-4cd7-8ced-9bf2d17716cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f8fd6b58b184bac824348b6a19dec4f'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:58.617997 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1068/2152] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:14:58 2022] DELETE /identity/v3/users/3f8fd6b58b184bac824348b6a19dec4f => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:14:58.633010 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72765084-185c-477a-bf42-5a699ff071d6 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:58.633864 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72765084-185c-477a-bf42-5a699ff071d6 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:58.634597 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72765084-185c-477a-bf42-5a699ff071d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:58.679001 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-72765084-185c-477a-bf42-5a699ff071d6 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:14:58.680694 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-72765084-185c-477a-bf42-5a699ff071d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:58.823442 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1085/2153] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:14:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:14:58.890659 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9b0742-ca63-4cd7-8ced-9bf2d17716cf admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:58.892319 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-ab9b0742-ca63-4cd7-8ced-9bf2d17716cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:59.039453 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9b0742-ca63-4cd7-8ced-9bf2d17716cf admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:59.050933 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6923d3b8-4eda-428a-b05e-47584db48908 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:59.052074 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923d3b8-4eda-428a-b05e-47584db48908 service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:59.052538 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923d3b8-4eda-428a-b05e-47584db48908 service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:59.052964 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923d3b8-4eda-428a-b05e-47584db48908 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:59.081028 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6923d3b8-4eda-428a-b05e-47584db48908 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:59.082891 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6923d3b8-4eda-428a-b05e-47584db48908 service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:59.103842 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1069/2154] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:14:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:14:59.737222 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72765084-185c-477a-bf42-5a699ff071d6 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:14:59.739886 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-72765084-185c-477a-bf42-5a699ff071d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:14:59.766222 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72765084-185c-477a-bf42-5a699ff071d6 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:14:59.784755 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ce1b33-774c-4dfc-a84d-3e3b8af6c010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:14:59.785396 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce1b33-774c-4dfc-a84d-3e3b8af6c010 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:14:59.785669 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce1b33-774c-4dfc-a84d-3e3b8af6c010 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:14:59.785949 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce1b33-774c-4dfc-a84d-3e3b8af6c010 admin admin] PATH_INFO: `/v3/projects/1dbe690577614c0e880bc13a3d659eb0` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:14:59.787361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ce1b33-774c-4dfc-a84d-3e3b8af6c010 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dbe690577614c0e880bc13a3d659eb0)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:14:59.789373 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ce1b33-774c-4dfc-a84d-3e3b8af6c010 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:14:59.791085 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:14:59.905910 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-03ce1b33-774c-4dfc-a84d-3e3b8af6c010 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dbe690577614c0e880bc13a3d659eb0', 'request_id': 'req-03ce1b33-774c-4dfc-a84d-3e3b8af6c010'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:59.924925 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-03ce1b33-774c-4dfc-a84d-3e3b8af6c010 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dbe690577614c0e880bc13a3d659eb0', 'request_id': 'req-03ce1b33-774c-4dfc-a84d-3e3b8af6c010'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:59.970559 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-03ce1b33-774c-4dfc-a84d-3e3b8af6c010 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dbe690577614c0e880bc13a3d659eb0', 'request_id': 'req-03ce1b33-774c-4dfc-a84d-3e3b8af6c010'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:59.998548 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-03ce1b33-774c-4dfc-a84d-3e3b8af6c010 admin admin] The token cache is being invalidate because project 1dbe690577614c0e880bc13a3d659eb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:14:59.998865 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-03ce1b33-774c-4dfc-a84d-3e3b8af6c010 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:14:59.999551 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-03ce1b33-774c-4dfc-a84d-3e3b8af6c010 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:15:00.001145 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1086/2155] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:14:59 2022] DELETE /identity/v3/projects/1dbe690577614c0e880bc13a3d659eb0 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:15:00.005176 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6923d3b8-4eda-428a-b05e-47584db48908 service neutron] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:00.018023 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6923d3b8-4eda-428a-b05e-47584db48908 service neutron] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:00.200168 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aab93d-e7a5-4a8b-9348-47d142aa6d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:00.200168 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aab93d-e7a5-4a8b-9348-47d142aa6d47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:00.200456 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aab93d-e7a5-4a8b-9348-47d142aa6d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:00.203626 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aab93d-e7a5-4a8b-9348-47d142aa6d47 admin admin] PATH_INFO: `/v3/users/bfc2ce8eda0a4b678be76603be19cfa3` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:00.207207 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aab93d-e7a5-4a8b-9348-47d142aa6d47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfc2ce8eda0a4b678be76603be19cfa3)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:00.225859 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aab93d-e7a5-4a8b-9348-47d142aa6d47 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:00.365072 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-e4aab93d-e7a5-4a8b-9348-47d142aa6d47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfc2ce8eda0a4b678be76603be19cfa3'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:15:00.392605 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-e4aab93d-e7a5-4a8b-9348-47d142aa6d47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfc2ce8eda0a4b678be76603be19cfa3'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:15:00.410206 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-e4aab93d-e7a5-4a8b-9348-47d142aa6d47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfc2ce8eda0a4b678be76603be19cfa3'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:15:00.411502 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-e4aab93d-e7a5-4a8b-9348-47d142aa6d47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfc2ce8eda0a4b678be76603be19cfa3'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:15:00.414994 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:15:00.433618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-e4aab93d-e7a5-4a8b-9348-47d142aa6d47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfc2ce8eda0a4b678be76603be19cfa3'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:15:00.446037 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1070/2156] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:15:00 2022] DELETE /identity/v3/users/bfc2ce8eda0a4b678be76603be19cfa3 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:15:01.738672 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ce1b33-774c-4dfc-a84d-3e3b8af6c010 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:01.740365 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-03ce1b33-774c-4dfc-a84d-3e3b8af6c010 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:15:01.772846 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ce1b33-774c-4dfc-a84d-3e3b8af6c010 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:01.933715 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a1c9ac-7705-4259-8140-5aacbe9f639c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:01.934454 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a1c9ac-7705-4259-8140-5aacbe9f639c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:01.938095 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a1c9ac-7705-4259-8140-5aacbe9f639c admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:01.938418 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a1c9ac-7705-4259-8140-5aacbe9f639c admin admin] PATH_INFO: `/v3/projects/6a74f378827c4b52916d8183c76382cc` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:01.941843 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a1c9ac-7705-4259-8140-5aacbe9f639c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a74f378827c4b52916d8183c76382cc)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:01.956758 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a1c9ac-7705-4259-8140-5aacbe9f639c admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:02.078243 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-e9a1c9ac-7705-4259-8140-5aacbe9f639c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a74f378827c4b52916d8183c76382cc', 'request_id': 'req-e9a1c9ac-7705-4259-8140-5aacbe9f639c'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:15:02.098023 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-e9a1c9ac-7705-4259-8140-5aacbe9f639c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a74f378827c4b52916d8183c76382cc', 'request_id': 'req-e9a1c9ac-7705-4259-8140-5aacbe9f639c'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:15:02.142845 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-e9a1c9ac-7705-4259-8140-5aacbe9f639c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a74f378827c4b52916d8183c76382cc', 'request_id': 'req-e9a1c9ac-7705-4259-8140-5aacbe9f639c'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:15:02.167062 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-e9a1c9ac-7705-4259-8140-5aacbe9f639c admin admin] The token cache is being invalidate because project 6a74f378827c4b52916d8183c76382cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:15:02.167354 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-e9a1c9ac-7705-4259-8140-5aacbe9f639c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:15:02.168076 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-e9a1c9ac-7705-4259-8140-5aacbe9f639c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:15:02.170669 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1087/2157] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:15:01 2022] DELETE /identity/v3/projects/6a74f378827c4b52916d8183c76382cc => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:15:02.176106 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a3532-4a74-42d6-bb62-807ad4359105 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:02.177465 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a3532-4a74-42d6-bb62-807ad4359105 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:02.177971 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a3532-4a74-42d6-bb62-807ad4359105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:02.236096 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-585a3532-4a74-42d6-bb62-807ad4359105 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:15:02.237404 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-585a3532-4a74-42d6-bb62-807ad4359105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:15:02.403757 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34e92f-85f6-4c60-969b-23f9cf64797c None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:02.404411 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34e92f-85f6-4c60-969b-23f9cf64797c None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:02.405006 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34e92f-85f6-4c60-969b-23f9cf64797c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:02.408987 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1071/2158] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:15:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:15:02.433689 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-1e34e92f-85f6-4c60-969b-23f9cf64797c None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:15:02.435971 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-1e34e92f-85f6-4c60-969b-23f9cf64797c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:15:02.453613 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585a3532-4a74-42d6-bb62-807ad4359105 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:02.461751 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1088/2159] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:15:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:15:02.469291 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585a3532-4a74-42d6-bb62-807ad4359105 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:02.483438 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e3d27c-592f-49e6-98c7-27c1c8d652c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:02.484470 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3d27c-592f-49e6-98c7-27c1c8d652c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:02.484824 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3d27c-592f-49e6-98c7-27c1c8d652c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:02.485262 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3d27c-592f-49e6-98c7-27c1c8d652c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:02.486064 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e3d27c-592f-49e6-98c7-27c1c8d652c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:15:02.486439 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e3d27c-592f-49e6-98c7-27c1c8d652c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:02.488155 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e3d27c-592f-49e6-98c7-27c1c8d652c1 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:02.502381 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1072/2160] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:15:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:15:02.506516 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e34e92f-85f6-4c60-969b-23f9cf64797c None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:02.519159 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e34e92f-85f6-4c60-969b-23f9cf64797c None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:02.524097 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675ded2a-2941-4a94-a35c-efbb7eada713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:02.524753 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ded2a-2941-4a94-a35c-efbb7eada713 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:02.525027 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ded2a-2941-4a94-a35c-efbb7eada713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:02.525358 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ded2a-2941-4a94-a35c-efbb7eada713 admin admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:02.526007 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675ded2a-2941-4a94-a35c-efbb7eada713 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:02.527924 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675ded2a-2941-4a94-a35c-efbb7eada713 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:02.543376 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:15:02.574141 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1089/2161] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:15:02 2022] POST /identity/v3/projects => generated 385 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 01:15:02.575948 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e3d27c-592f-49e6-98c7-27c1c8d652c1 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:02.600272 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e3d27c-592f-49e6-98c7-27c1c8d652c1 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:02.774707 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da03110a-3722-4ae3-96d6-ac3ca55f56a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:02.775988 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03110a-3722-4ae3-96d6-ac3ca55f56a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:02.776373 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03110a-3722-4ae3-96d6-ac3ca55f56a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:02.776724 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03110a-3722-4ae3-96d6-ac3ca55f56a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:02.777848 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da03110a-3722-4ae3-96d6-ac3ca55f56a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:02.785650 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da03110a-3722-4ae3-96d6-ac3ca55f56a9 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:02.831919 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1073/2162] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:15:02 2022] POST /identity/v3/users => generated 293 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:15:02.835979 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675ded2a-2941-4a94-a35c-efbb7eada713 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:02.854486 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675ded2a-2941-4a94-a35c-efbb7eada713 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:02.876951 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068973d6-656d-4702-bfd5-b038ca658fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:02.877808 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068973d6-656d-4702-bfd5-b038ca658fc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:02.878087 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068973d6-656d-4702-bfd5-b038ca658fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:02.878518 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068973d6-656d-4702-bfd5-b038ca658fc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:02.879317 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068973d6-656d-4702-bfd5-b038ca658fc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:02.880990 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068973d6-656d-4702-bfd5-b038ca658fc3 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:02.904003 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1090/2163] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:15:02 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:15:02.908653 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da03110a-3722-4ae3-96d6-ac3ca55f56a9 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:02.927017 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da03110a-3722-4ae3-96d6-ac3ca55f56a9 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:02.948427 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfa4a81-262b-4d5a-bed4-cc439215b29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:02.949060 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfa4a81-262b-4d5a-bed4-cc439215b29c admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:02.949344 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfa4a81-262b-4d5a-bed4-cc439215b29c admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:02.949619 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfa4a81-262b-4d5a-bed4-cc439215b29c admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:02.950281 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfa4a81-262b-4d5a-bed4-cc439215b29c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:02.952539 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfa4a81-262b-4d5a-bed4-cc439215b29c admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:02.977294 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1074/2164] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:15:02 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:15:02.981209 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068973d6-656d-4702-bfd5-b038ca658fc3 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:02.993794 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068973d6-656d-4702-bfd5-b038ca658fc3 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:03.015677 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd7e68f-8bef-433c-97da-d74d03580da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:03.016394 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd7e68f-8bef-433c-97da-d74d03580da3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:03.016679 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd7e68f-8bef-433c-97da-d74d03580da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:03.016957 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd7e68f-8bef-433c-97da-d74d03580da3 admin admin] PATH_INFO: `/v3/projects/c975e3fefb804aacae39527c3fa01d63/users/73c0c30efaa9413da3b97e813f957ec5/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:03.045970 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd7e68f-8bef-433c-97da-d74d03580da3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c975e3fefb804aacae39527c3fa01d63, user_id=73c0c30efaa9413da3b97e813f957ec5, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:03.048741 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd7e68f-8bef-433c-97da-d74d03580da3 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:03.079976 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1091/2165] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:15:02 2022] PUT /identity/v3/projects/c975e3fefb804aacae39527c3fa01d63/users/73c0c30efaa9413da3b97e813f957ec5/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:15:03.091135 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84160b-a7ba-415b-b354-0afe45be33d1 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:03.091449 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84160b-a7ba-415b-b354-0afe45be33d1 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:03.091731 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84160b-a7ba-415b-b354-0afe45be33d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:03.158158 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-3d84160b-a7ba-415b-b354-0afe45be33d1 None None] MFA Rules not processed for user `73c0c30efaa9413da3b97e813f957ec5`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:15:03.159478 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-3d84160b-a7ba-415b-b354-0afe45be33d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:15:03.281972 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:15:03.331474 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1075/2166] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:15:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:15:03.336039 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd7e68f-8bef-433c-97da-d74d03580da3 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:03.501858 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd7e68f-8bef-433c-97da-d74d03580da3 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:03.514412 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ddd26f-db49-4647-a8fd-d29401755c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:03.515759 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ddd26f-db49-4647-a8fd-d29401755c9f admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:03.516254 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ddd26f-db49-4647-a8fd-d29401755c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:03.516727 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ddd26f-db49-4647-a8fd-d29401755c9f admin admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:03.517559 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ddd26f-db49-4647-a8fd-d29401755c9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:03.519717 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ddd26f-db49-4647-a8fd-d29401755c9f admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:03.582029 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1092/2167] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:15:03 2022] POST /identity/v3/projects => generated 387 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:15:03.586571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d84160b-a7ba-415b-b354-0afe45be33d1 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:03.598836 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d84160b-a7ba-415b-b354-0afe45be33d1 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:03.691907 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9007643-3298-486a-9458-e03b1329ca70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:03.693556 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9007643-3298-486a-9458-e03b1329ca70 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:03.693915 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9007643-3298-486a-9458-e03b1329ca70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:03.694283 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9007643-3298-486a-9458-e03b1329ca70 admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:03.695806 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9007643-3298-486a-9458-e03b1329ca70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:03.698299 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9007643-3298-486a-9458-e03b1329ca70 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:03.727776 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1076/2168] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:15:03 2022] POST /identity/v3/users => generated 300 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:15:03.732936 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ddd26f-db49-4647-a8fd-d29401755c9f admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:03.747005 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ddd26f-db49-4647-a8fd-d29401755c9f admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:03.753704 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7c281-8f4e-4e93-a14e-91419c6157ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:03.754614 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7c281-8f4e-4e93-a14e-91419c6157ce admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:03.755061 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7c281-8f4e-4e93-a14e-91419c6157ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:03.772468 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7c281-8f4e-4e93-a14e-91419c6157ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:03.774808 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac7c281-8f4e-4e93-a14e-91419c6157ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:03.780605 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac7c281-8f4e-4e93-a14e-91419c6157ce admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:03.801814 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1093/2169] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:15:03 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:15:03.805912 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9007643-3298-486a-9458-e03b1329ca70 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:03.820386 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9007643-3298-486a-9458-e03b1329ca70 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:03.828183 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a445fe25-ddb3-40e2-927b-917ca65d2215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:03.828933 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a445fe25-ddb3-40e2-927b-917ca65d2215 admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:03.829217 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a445fe25-ddb3-40e2-927b-917ca65d2215 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:03.829738 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a445fe25-ddb3-40e2-927b-917ca65d2215 admin admin] PATH_INFO: `/v3/projects/f9bf00521dc74df4bfeb59bd7f0ca180/users/11a201f0cadb45a59f20a248616ba5c9/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:03.855143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a445fe25-ddb3-40e2-927b-917ca65d2215 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9bf00521dc74df4bfeb59bd7f0ca180, user_id=11a201f0cadb45a59f20a248616ba5c9, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:03.862174 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a445fe25-ddb3-40e2-927b-917ca65d2215 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:03.880966 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1077/2170] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:15:03 2022] PUT /identity/v3/projects/f9bf00521dc74df4bfeb59bd7f0ca180/users/11a201f0cadb45a59f20a248616ba5c9/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:15:03.885072 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7c281-8f4e-4e93-a14e-91419c6157ce admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:03.898779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7c281-8f4e-4e93-a14e-91419c6157ce admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:03.994430 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f10580c-23c9-4b82-a196-24cb4d075cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:03.995241 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f10580c-23c9-4b82-a196-24cb4d075cfa admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:03.996465 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f10580c-23c9-4b82-a196-24cb4d075cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:03.996932 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f10580c-23c9-4b82-a196-24cb4d075cfa admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:03.998124 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f10580c-23c9-4b82-a196-24cb4d075cfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:04.000309 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f10580c-23c9-4b82-a196-24cb4d075cfa admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:04.030200 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1094/2171] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:15:03 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:15:04.033896 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a445fe25-ddb3-40e2-927b-917ca65d2215 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:04.055299 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a445fe25-ddb3-40e2-927b-917ca65d2215 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:04.072084 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd06e6-f58c-441b-9923-aed4239d7336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:04.073094 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd06e6-f58c-441b-9923-aed4239d7336 admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:04.073452 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd06e6-f58c-441b-9923-aed4239d7336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:04.073736 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd06e6-f58c-441b-9923-aed4239d7336 admin admin] PATH_INFO: `/v3/projects/f9bf00521dc74df4bfeb59bd7f0ca180/users/11a201f0cadb45a59f20a248616ba5c9/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:04.077549 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cd06e6-f58c-441b-9923-aed4239d7336 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9bf00521dc74df4bfeb59bd7f0ca180, user_id=11a201f0cadb45a59f20a248616ba5c9, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:04.080475 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cd06e6-f58c-441b-9923-aed4239d7336 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:04.109491 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1078/2172] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:15:04 2022] PUT /identity/v3/projects/f9bf00521dc74df4bfeb59bd7f0ca180/users/11a201f0cadb45a59f20a248616ba5c9/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:15:04.119463 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f167f1e-528b-4db9-9894-a43d9ec2414b None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:04.119763 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f167f1e-528b-4db9-9894-a43d9ec2414b None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:04.120038 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f167f1e-528b-4db9-9894-a43d9ec2414b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:04.229521 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-3f167f1e-528b-4db9-9894-a43d9ec2414b None None] MFA Rules not processed for user `11a201f0cadb45a59f20a248616ba5c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:15:04.230982 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-3f167f1e-528b-4db9-9894-a43d9ec2414b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:15:04.248684 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:15:04.377408 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1095/2173] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:15:04 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:15:04.389580 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd06e6-f58c-441b-9923-aed4239d7336 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:04.391436 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-b9cd06e6-f58c-441b-9923-aed4239d7336 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:15:04.547946 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd06e6-f58c-441b-9923-aed4239d7336 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:04.558702 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679ac8a5-00a6-45b2-8b8d-d88e6e4ccdcd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:04.559601 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ac8a5-00a6-45b2-8b8d-d88e6e4ccdcd service cinder] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:04.560160 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ac8a5-00a6-45b2-8b8d-d88e6e4ccdcd service cinder] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:04.560249 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ac8a5-00a6-45b2-8b8d-d88e6e4ccdcd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:04.585726 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679ac8a5-00a6-45b2-8b8d-d88e6e4ccdcd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:04.587864 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679ac8a5-00a6-45b2-8b8d-d88e6e4ccdcd service cinder] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:04.614322 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1079/2174] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:15:04 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:15:04.660879 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2887d1-acab-4bbd-bc46-e9b9a785f22c None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:04.661429 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2887d1-acab-4bbd-bc46-e9b9a785f22c None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:04.662709 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2887d1-acab-4bbd-bc46-e9b9a785f22c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:04.698629 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-2c2887d1-acab-4bbd-bc46-e9b9a785f22c None None] MFA Rules not processed for user `dc29e7a08e1d4f0fbd259ee49b4a97d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:15:04.700105 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-2c2887d1-acab-4bbd-bc46-e9b9a785f22c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:15:04.723218 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679ac8a5-00a6-45b2-8b8d-d88e6e4ccdcd service cinder] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:04.837367 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:15:04.854641 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1096/2175] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:15:04 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:15:04.867606 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2887d1-acab-4bbd-bc46-e9b9a785f22c None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:04.869668 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-2c2887d1-acab-4bbd-bc46-e9b9a785f22c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:15:04.884243 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679ac8a5-00a6-45b2-8b8d-d88e6e4ccdcd service cinder] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:04.898291 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb9770-356d-4e05-9e89-ead4900e196d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:04.900095 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb9770-356d-4e05-9e89-ead4900e196d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:04.904125 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb9770-356d-4e05-9e89-ead4900e196d admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:04.904836 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb9770-356d-4e05-9e89-ead4900e196d admin admin] PATH_INFO: `/v3/users/73c0c30efaa9413da3b97e813f957ec5` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:04.909061 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcb9770-356d-4e05-9e89-ead4900e196d admin admin] RBAC: Authorizing `identity:delete_user(user_id=73c0c30efaa9413da3b97e813f957ec5)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:04.912072 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcb9770-356d-4e05-9e89-ead4900e196d admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:05.015962 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2887d1-acab-4bbd-bc46-e9b9a785f22c None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:05.024752 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-7fcb9770-356d-4e05-9e89-ead4900e196d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73c0c30efaa9413da3b97e813f957ec5'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:15:05.027087 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea94b21-8d01-47df-99af-c5389fc9cab6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1acfa7a09d34e2a9045c3f3fbea0f52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:05.027711 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea94b21-8d01-47df-99af-c5389fc9cab6 service glance] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:05.027985 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea94b21-8d01-47df-99af-c5389fc9cab6 service glance] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:05.030481 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea94b21-8d01-47df-99af-c5389fc9cab6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:05.036523 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-cea94b21-8d01-47df-99af-c5389fc9cab6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:15:05.060761 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-7fcb9770-356d-4e05-9e89-ead4900e196d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73c0c30efaa9413da3b97e813f957ec5'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:15:05.081026 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-7fcb9770-356d-4e05-9e89-ead4900e196d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73c0c30efaa9413da3b97e813f957ec5'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:15:05.081795 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-7fcb9770-356d-4e05-9e89-ead4900e196d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73c0c30efaa9413da3b97e813f957ec5'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:15:05.103265 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-7fcb9770-356d-4e05-9e89-ead4900e196d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73c0c30efaa9413da3b97e813f957ec5'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:15:05.116334 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1080/2176] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:15:04 2022] DELETE /identity/v3/users/73c0c30efaa9413da3b97e813f957ec5 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:15:05.118306 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c6f93d-5c6e-4d8e-87f1-a0a292bd78d8 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:05.119457 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c6f93d-5c6e-4d8e-87f1-a0a292bd78d8 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:05.119884 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c6f93d-5c6e-4d8e-87f1-a0a292bd78d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:05.147852 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-e2c6f93d-5c6e-4d8e-87f1-a0a292bd78d8 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:15:05.149464 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-e2c6f93d-5c6e-4d8e-87f1-a0a292bd78d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:15:05.173485 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:15:05.193424 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea94b21-8d01-47df-99af-c5389fc9cab6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:05.195091 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea94b21-8d01-47df-99af-c5389fc9cab6 service glance] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:05.238002 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1097/2177] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:15:04 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:15:05.274334 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1081/2178] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:15:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:15:05.286709 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea94b21-8d01-47df-99af-c5389fc9cab6 service glance] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:05.288315 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-cea94b21-8d01-47df-99af-c5389fc9cab6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:15:05.424562 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea94b21-8d01-47df-99af-c5389fc9cab6 service glance] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:05.545084 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a80e17-e677-4fe8-8538-1c293863f310 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:05.545562 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a80e17-e677-4fe8-8538-1c293863f310 service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:05.545844 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a80e17-e677-4fe8-8538-1c293863f310 service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:05.546148 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a80e17-e677-4fe8-8538-1c293863f310 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:05.567515 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a80e17-e677-4fe8-8538-1c293863f310 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:05.569681 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a80e17-e677-4fe8-8538-1c293863f310 service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:05.603089 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1098/2179] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:15:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:15:05.662117 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e06525-6156-47fa-ad9e-448a51edb2c5 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:05.662977 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e06525-6156-47fa-ad9e-448a51edb2c5 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:05.663420 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e06525-6156-47fa-ad9e-448a51edb2c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:05.701058 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-14e06525-6156-47fa-ad9e-448a51edb2c5 None None] MFA Rules not processed for user `de75fd7d7721433684cafa2e6d3faaae`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:15:05.702636 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-14e06525-6156-47fa-ad9e-448a51edb2c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:15:05.702831 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a886f-cde2-4b5e-9441-43986231b91f None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:05.703119 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a886f-cde2-4b5e-9441-43986231b91f None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:05.703416 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a886f-cde2-4b5e-9441-43986231b91f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:05.760355 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-4d7a886f-cde2-4b5e-9441-43986231b91f None None] MFA Rules not processed for user `83fea9762aaf4766a23f1774ff1411fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:15:05.762130 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-4d7a886f-cde2-4b5e-9441-43986231b91f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:15:05.817256 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1082/2180] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Sep 8 01:15:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:15:05.898982 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e06525-6156-47fa-ad9e-448a51edb2c5 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:05.901096 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-14e06525-6156-47fa-ad9e-448a51edb2c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:15:05.918049 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1099/2181] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:15:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:15:05.919133 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7a886f-cde2-4b5e-9441-43986231b91f None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:05.920055 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:15:05.925221 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-4d7a886f-cde2-4b5e-9441-43986231b91f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:15:06.057562 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:15:06.102331 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e06525-6156-47fa-ad9e-448a51edb2c5 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:06.111841 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65552ca3-b8a7-450d-b2cf-318e30390630 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8add26872fe6479aa9d7de155a17195a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:06.112772 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7a886f-cde2-4b5e-9441-43986231b91f None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:06.112956 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65552ca3-b8a7-450d-b2cf-318e30390630 service swift] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:06.113267 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65552ca3-b8a7-450d-b2cf-318e30390630 service swift] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:06.113542 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65552ca3-b8a7-450d-b2cf-318e30390630 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:06.118984 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b6c86-663f-4d82-ab47-687d1706601f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae643cfdcb164aa9b474747cd21aa116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:06.119733 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b6c86-663f-4d82-ab47-687d1706601f service nova] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:06.120019 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b6c86-663f-4d82-ab47-687d1706601f service nova] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:06.120357 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b6c86-663f-4d82-ab47-687d1706601f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:06.127622 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65552ca3-b8a7-450d-b2cf-318e30390630 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:06.132221 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65552ca3-b8a7-450d-b2cf-318e30390630 service swift] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:06.145645 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1083/2182] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Sep 8 01:15:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 08 01:15:06.146703 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65552ca3-b8a7-450d-b2cf-318e30390630 service swift] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:06.148077 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3b6c86-663f-4d82-ab47-687d1706601f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:06.149476 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-65552ca3-b8a7-450d-b2cf-318e30390630 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:15:06.149860 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3b6c86-663f-4d82-ab47-687d1706601f service nova] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:06.176560 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65552ca3-b8a7-450d-b2cf-318e30390630 service swift] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:06.185459 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb7a40-6b95-40cb-a817-86a281e17c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:06.186297 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb7a40-6b95-40cb-a817-86a281e17c69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:06.186748 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb7a40-6b95-40cb-a817-86a281e17c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:06.187165 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb7a40-6b95-40cb-a817-86a281e17c69 admin admin] PATH_INFO: `/v3/projects/c975e3fefb804aacae39527c3fa01d63` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:06.188790 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bb7a40-6b95-40cb-a817-86a281e17c69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c975e3fefb804aacae39527c3fa01d63)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:06.191312 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bb7a40-6b95-40cb-a817-86a281e17c69 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:06.220651 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1100/2183] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:15:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:15:06.321552 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-d1bb7a40-6b95-40cb-a817-86a281e17c69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c975e3fefb804aacae39527c3fa01d63', 'request_id': 'req-d1bb7a40-6b95-40cb-a817-86a281e17c69'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:15:06.343617 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-d1bb7a40-6b95-40cb-a817-86a281e17c69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c975e3fefb804aacae39527c3fa01d63', 'request_id': 'req-d1bb7a40-6b95-40cb-a817-86a281e17c69'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:15:06.353550 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-d1bb7a40-6b95-40cb-a817-86a281e17c69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c975e3fefb804aacae39527c3fa01d63', 'request_id': 'req-d1bb7a40-6b95-40cb-a817-86a281e17c69'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:15:06.367982 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-d1bb7a40-6b95-40cb-a817-86a281e17c69 admin admin] The token cache is being invalidate because project c975e3fefb804aacae39527c3fa01d63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81661) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:15:06.368325 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-d1bb7a40-6b95-40cb-a817-86a281e17c69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:15:06.369375 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-d1bb7a40-6b95-40cb-a817-86a281e17c69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:15:06.372008 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1084/2184] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:15:06 2022] DELETE /identity/v3/projects/c975e3fefb804aacae39527c3fa01d63 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:15:06.376037 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b6c86-663f-4d82-ab47-687d1706601f service nova] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:06.391991 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b6c86-663f-4d82-ab47-687d1706601f service nova] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:06.473872 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee0b22a-bd71-486f-86cc-b3adeff9188b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:06.474640 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0b22a-bd71-486f-86cc-b3adeff9188b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:06.474916 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0b22a-bd71-486f-86cc-b3adeff9188b admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:06.475188 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0b22a-bd71-486f-86cc-b3adeff9188b admin admin] PATH_INFO: `/v3/users/11a201f0cadb45a59f20a248616ba5c9` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:06.476437 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee0b22a-bd71-486f-86cc-b3adeff9188b admin admin] RBAC: Authorizing `identity:delete_user(user_id=11a201f0cadb45a59f20a248616ba5c9)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:06.478682 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee0b22a-bd71-486f-86cc-b3adeff9188b admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:06.585617 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-aee0b22a-bd71-486f-86cc-b3adeff9188b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11a201f0cadb45a59f20a248616ba5c9'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:15:06.609257 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-aee0b22a-bd71-486f-86cc-b3adeff9188b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11a201f0cadb45a59f20a248616ba5c9'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:15:06.625419 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-aee0b22a-bd71-486f-86cc-b3adeff9188b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11a201f0cadb45a59f20a248616ba5c9'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:15:06.640649 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-aee0b22a-bd71-486f-86cc-b3adeff9188b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11a201f0cadb45a59f20a248616ba5c9'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:15:06.642958 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-aee0b22a-bd71-486f-86cc-b3adeff9188b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11a201f0cadb45a59f20a248616ba5c9'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:15:06.662192 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1101/2185] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:15:06 2022] DELETE /identity/v3/users/11a201f0cadb45a59f20a248616ba5c9 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:15:07.327860 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb7a40-6b95-40cb-a817-86a281e17c69 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:07.329450 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-d1bb7a40-6b95-40cb-a817-86a281e17c69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:15:07.362691 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb7a40-6b95-40cb-a817-86a281e17c69 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:07.474877 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815ce98f-851e-4423-a8c2-39fa4eeca4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:07.475824 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ce98f-851e-4423-a8c2-39fa4eeca4ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:07.476331 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ce98f-851e-4423-a8c2-39fa4eeca4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:07.476770 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ce98f-851e-4423-a8c2-39fa4eeca4ef admin admin] PATH_INFO: `/v3/projects/f9bf00521dc74df4bfeb59bd7f0ca180` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:07.478873 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815ce98f-851e-4423-a8c2-39fa4eeca4ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9bf00521dc74df4bfeb59bd7f0ca180)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:07.483835 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815ce98f-851e-4423-a8c2-39fa4eeca4ef admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:07.596321 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:15:07.729228 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-815ce98f-851e-4423-a8c2-39fa4eeca4ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9bf00521dc74df4bfeb59bd7f0ca180', 'request_id': 'req-815ce98f-851e-4423-a8c2-39fa4eeca4ef'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:15:07.750902 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-815ce98f-851e-4423-a8c2-39fa4eeca4ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9bf00521dc74df4bfeb59bd7f0ca180', 'request_id': 'req-815ce98f-851e-4423-a8c2-39fa4eeca4ef'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:15:07.759293 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-815ce98f-851e-4423-a8c2-39fa4eeca4ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9bf00521dc74df4bfeb59bd7f0ca180', 'request_id': 'req-815ce98f-851e-4423-a8c2-39fa4eeca4ef'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:15:07.791136 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-815ce98f-851e-4423-a8c2-39fa4eeca4ef admin admin] The token cache is being invalidate because project f9bf00521dc74df4bfeb59bd7f0ca180 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81661) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:15:07.791512 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-815ce98f-851e-4423-a8c2-39fa4eeca4ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:15:07.796519 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-815ce98f-851e-4423-a8c2-39fa4eeca4ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:15:07.801702 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1085/2186] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:15:07 2022] DELETE /identity/v3/projects/f9bf00521dc74df4bfeb59bd7f0ca180 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:15:07.803585 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73d1df-0c7a-40e5-a9e5-03131366c190 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:07.804175 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73d1df-0c7a-40e5-a9e5-03131366c190 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:07.804175 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73d1df-0c7a-40e5-a9e5-03131366c190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:07.848496 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-3f73d1df-0c7a-40e5-a9e5-03131366c190 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:15:07.850045 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-3f73d1df-0c7a-40e5-a9e5-03131366c190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:15:07.983063 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:15:08.041643 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1102/2187] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:15:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:15:08.051282 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b005e52-ff67-45a5-838f-efa63b2be204 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:08.051757 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b005e52-ff67-45a5-838f-efa63b2be204 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:08.052191 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b005e52-ff67-45a5-838f-efa63b2be204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:08.098182 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-1b005e52-ff67-45a5-838f-efa63b2be204 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:15:08.101158 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-1b005e52-ff67-45a5-838f-efa63b2be204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:15:08.123304 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1086/2188] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:15:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:15:08.128305 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f73d1df-0c7a-40e5-a9e5-03131366c190 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:08.140089 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f73d1df-0c7a-40e5-a9e5-03131366c190 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:08.148887 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f1953e-a2b6-42e7-b44d-599080eab01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:08.149928 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f1953e-a2b6-42e7-b44d-599080eab01b admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:08.150428 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f1953e-a2b6-42e7-b44d-599080eab01b admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:08.150939 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f1953e-a2b6-42e7-b44d-599080eab01b admin admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:08.151838 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f1953e-a2b6-42e7-b44d-599080eab01b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:15:08.152434 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f1953e-a2b6-42e7-b44d-599080eab01b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:08.157405 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f1953e-a2b6-42e7-b44d-599080eab01b admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:08.177202 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1103/2189] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:15:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:15:08.185846 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b005e52-ff67-45a5-838f-efa63b2be204 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:08.197999 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b005e52-ff67-45a5-838f-efa63b2be204 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:08.209447 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf97a67-cf4f-43ae-9f2e-d22f521ba80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:08.210093 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf97a67-cf4f-43ae-9f2e-d22f521ba80e admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:08.210376 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf97a67-cf4f-43ae-9f2e-d22f521ba80e admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:08.210648 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf97a67-cf4f-43ae-9f2e-d22f521ba80e admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:08.211351 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf97a67-cf4f-43ae-9f2e-d22f521ba80e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:08.214948 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf97a67-cf4f-43ae-9f2e-d22f521ba80e admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:08.360242 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1087/2190] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:15:08 2022] POST /identity/v3/projects => generated 385 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:15:08.362397 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f1953e-a2b6-42e7-b44d-599080eab01b admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:08.374549 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f1953e-a2b6-42e7-b44d-599080eab01b admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:08.615252 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621e40a1-6d88-44bd-bae1-ed8394eb84d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:08.616279 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e40a1-6d88-44bd-bae1-ed8394eb84d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:08.616925 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e40a1-6d88-44bd-bae1-ed8394eb84d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:08.626997 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e40a1-6d88-44bd-bae1-ed8394eb84d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:08.630042 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621e40a1-6d88-44bd-bae1-ed8394eb84d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:08.634516 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621e40a1-6d88-44bd-bae1-ed8394eb84d2 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:08.712313 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1104/2191] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:15:08 2022] POST /identity/v3/users => generated 293 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 01:15:08.713512 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf97a67-cf4f-43ae-9f2e-d22f521ba80e admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:08.737716 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf97a67-cf4f-43ae-9f2e-d22f521ba80e admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:08.757000 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dbf19c-cd33-4bb5-86b3-a78ebf4f4369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:08.761265 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dbf19c-cd33-4bb5-86b3-a78ebf4f4369 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:08.761663 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dbf19c-cd33-4bb5-86b3-a78ebf4f4369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:08.762490 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dbf19c-cd33-4bb5-86b3-a78ebf4f4369 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:08.764669 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dbf19c-cd33-4bb5-86b3-a78ebf4f4369 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:08.772523 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dbf19c-cd33-4bb5-86b3-a78ebf4f4369 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:08.795093 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621e40a1-6d88-44bd-bae1-ed8394eb84d2 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:08.807322 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621e40a1-6d88-44bd-bae1-ed8394eb84d2 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:08.808469 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1088/2192] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:15:08.820513 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97fbfcd-2417-46fb-85f7-6ad1dd855bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:08.821519 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97fbfcd-2417-46fb-85f7-6ad1dd855bac admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:08.822005 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97fbfcd-2417-46fb-85f7-6ad1dd855bac admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:08.822477 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97fbfcd-2417-46fb-85f7-6ad1dd855bac admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:08.823329 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97fbfcd-2417-46fb-85f7-6ad1dd855bac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:08.825585 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97fbfcd-2417-46fb-85f7-6ad1dd855bac admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:08.862684 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1105/2193] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:15:08.865224 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dbf19c-cd33-4bb5-86b3-a78ebf4f4369 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:08.898789 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dbf19c-cd33-4bb5-86b3-a78ebf4f4369 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:08.907310 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d5ef05-378b-4e28-8172-7fc47060ed30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:08.910312 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d5ef05-378b-4e28-8172-7fc47060ed30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:08.910601 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d5ef05-378b-4e28-8172-7fc47060ed30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:08.910918 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d5ef05-378b-4e28-8172-7fc47060ed30 admin admin] PATH_INFO: `/v3/projects/1f423a5ca0a4413496b3aaa36db0d4bf/users/7863cbc543ab4b908702747a0befe4f9/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:08.971719 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d5ef05-378b-4e28-8172-7fc47060ed30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f423a5ca0a4413496b3aaa36db0d4bf, user_id=7863cbc543ab4b908702747a0befe4f9, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:08.977047 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d5ef05-378b-4e28-8172-7fc47060ed30 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:09.000396 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1089/2194] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:15:08 2022] PUT /identity/v3/projects/1f423a5ca0a4413496b3aaa36db0d4bf/users/7863cbc543ab4b908702747a0befe4f9/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:15:09.011013 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb2595b-0640-44ca-953e-3161dde08cb2 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:09.011433 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb2595b-0640-44ca-953e-3161dde08cb2 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:09.011961 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb2595b-0640-44ca-953e-3161dde08cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:09.058797 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-dcb2595b-0640-44ca-953e-3161dde08cb2 None None] MFA Rules not processed for user `7863cbc543ab4b908702747a0befe4f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:15:09.060557 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-dcb2595b-0640-44ca-953e-3161dde08cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:15:09.096627 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf4f32a-3e16-4d56-ac2e-3bfb1aceebd3 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:09.097392 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf4f32a-3e16-4d56-ac2e-3bfb1aceebd3 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:09.097902 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf4f32a-3e16-4d56-ac2e-3bfb1aceebd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:09.130328 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-adf4f32a-3e16-4d56-ac2e-3bfb1aceebd3 None None] MFA Rules not processed for user `83fea9762aaf4766a23f1774ff1411fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:15:09.134341 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-adf4f32a-3e16-4d56-ac2e-3bfb1aceebd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:15:09.188972 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1106/2195] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:15:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:15:09.193489 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb2595b-0640-44ca-953e-3161dde08cb2 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:09.213377 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-dcb2595b-0640-44ca-953e-3161dde08cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:15:09.260849 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1090/2196] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:15:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:15:09.349445 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:15:09.377582 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb2595b-0640-44ca-953e-3161dde08cb2 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:09.381452 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9452aa7-7bee-4555-9ce3-4898fc1338d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:09.383060 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9452aa7-7bee-4555-9ce3-4898fc1338d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:09.383452 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9452aa7-7bee-4555-9ce3-4898fc1338d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:09.383830 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9452aa7-7bee-4555-9ce3-4898fc1338d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:09.406293 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9452aa7-7bee-4555-9ce3-4898fc1338d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:09.407859 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9452aa7-7bee-4555-9ce3-4898fc1338d7 service cinder] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:09.418877 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1107/2197] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:15:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:15:09.442306 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf4f32a-3e16-4d56-ac2e-3bfb1aceebd3 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:09.524200 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:15:09.539761 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf4f32a-3e16-4d56-ac2e-3bfb1aceebd3 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:09.546939 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e785b28-386d-48b3-9521-e8a32b8b6688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:09.548585 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e785b28-386d-48b3-9521-e8a32b8b6688 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:09.548585 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e785b28-386d-48b3-9521-e8a32b8b6688 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:09.548585 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e785b28-386d-48b3-9521-e8a32b8b6688 admin admin] PATH_INFO: `/v3/users/7863cbc543ab4b908702747a0befe4f9` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:09.550174 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e785b28-386d-48b3-9521-e8a32b8b6688 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7863cbc543ab4b908702747a0befe4f9)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:09.559027 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e785b28-386d-48b3-9521-e8a32b8b6688 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:09.664351 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-5e785b28-386d-48b3-9521-e8a32b8b6688 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7863cbc543ab4b908702747a0befe4f9'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:15:09.687629 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-5e785b28-386d-48b3-9521-e8a32b8b6688 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7863cbc543ab4b908702747a0befe4f9'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:15:09.707201 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-5e785b28-386d-48b3-9521-e8a32b8b6688 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7863cbc543ab4b908702747a0befe4f9'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:15:09.710658 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-5e785b28-386d-48b3-9521-e8a32b8b6688 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7863cbc543ab4b908702747a0befe4f9'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:15:09.726884 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-5e785b28-386d-48b3-9521-e8a32b8b6688 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7863cbc543ab4b908702747a0befe4f9'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:15:09.745109 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1091/2198] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:15:09 2022] DELETE /identity/v3/users/7863cbc543ab4b908702747a0befe4f9 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:15:09.748558 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81868b1-c4e5-4494-9055-48733ba25f69 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:09.749336 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81868b1-c4e5-4494-9055-48733ba25f69 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:09.749807 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81868b1-c4e5-4494-9055-48733ba25f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:09.775399 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-f81868b1-c4e5-4494-9055-48733ba25f69 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:15:09.776968 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-f81868b1-c4e5-4494-9055-48733ba25f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:15:09.949435 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1108/2199] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:15:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:15:09.950658 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e785b28-386d-48b3-9521-e8a32b8b6688 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:09.951686 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-5e785b28-386d-48b3-9521-e8a32b8b6688 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:15:10.085672 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e785b28-386d-48b3-9521-e8a32b8b6688 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:10.090111 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6311f7-9fa3-43dc-a1ad-c4df71d0a1ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:10.090809 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6311f7-9fa3-43dc-a1ad-c4df71d0a1ac service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:10.091101 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6311f7-9fa3-43dc-a1ad-c4df71d0a1ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:10.091545 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6311f7-9fa3-43dc-a1ad-c4df71d0a1ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:10.111949 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6311f7-9fa3-43dc-a1ad-c4df71d0a1ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:10.115063 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6311f7-9fa3-43dc-a1ad-c4df71d0a1ac service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:10.146729 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1092/2200] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:15:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 01:15:10.794703 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81868b1-c4e5-4494-9055-48733ba25f69 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:10.796026 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-f81868b1-c4e5-4494-9055-48733ba25f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:15:10.817171 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81868b1-c4e5-4494-9055-48733ba25f69 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:10.828124 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f57c35f-1939-4779-a0f4-7e6812d5bd2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:10.829039 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57c35f-1939-4779-a0f4-7e6812d5bd2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:10.829467 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57c35f-1939-4779-a0f4-7e6812d5bd2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:10.829912 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57c35f-1939-4779-a0f4-7e6812d5bd2a admin admin] PATH_INFO: `/v3/projects/1f423a5ca0a4413496b3aaa36db0d4bf` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:10.832671 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f57c35f-1939-4779-a0f4-7e6812d5bd2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f423a5ca0a4413496b3aaa36db0d4bf)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:10.834785 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f57c35f-1939-4779-a0f4-7e6812d5bd2a admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:10.993581 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-9f57c35f-1939-4779-a0f4-7e6812d5bd2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f423a5ca0a4413496b3aaa36db0d4bf', 'request_id': 'req-9f57c35f-1939-4779-a0f4-7e6812d5bd2a'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:15:11.008270 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-9f57c35f-1939-4779-a0f4-7e6812d5bd2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f423a5ca0a4413496b3aaa36db0d4bf', 'request_id': 'req-9f57c35f-1939-4779-a0f4-7e6812d5bd2a'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:15:11.024250 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-9f57c35f-1939-4779-a0f4-7e6812d5bd2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f423a5ca0a4413496b3aaa36db0d4bf', 'request_id': 'req-9f57c35f-1939-4779-a0f4-7e6812d5bd2a'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:15:11.055749 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-9f57c35f-1939-4779-a0f4-7e6812d5bd2a admin admin] The token cache is being invalidate because project 1f423a5ca0a4413496b3aaa36db0d4bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:15:11.056180 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-9f57c35f-1939-4779-a0f4-7e6812d5bd2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:15:11.058174 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-9f57c35f-1939-4779-a0f4-7e6812d5bd2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:15:11.063841 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1109/2201] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:15:10 2022] DELETE /identity/v3/projects/1f423a5ca0a4413496b3aaa36db0d4bf => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:15:11.078065 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3a270-4f92-477f-956d-6fad505724f3 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:11.078475 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3a270-4f92-477f-956d-6fad505724f3 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:11.078475 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3a270-4f92-477f-956d-6fad505724f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:11.136483 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-64f3a270-4f92-477f-956d-6fad505724f3 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:15:11.141031 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-64f3a270-4f92-477f-956d-6fad505724f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:15:11.241538 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1093/2202] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:15:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:15:11.251947 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1804732-cbb8-436e-808f-8458f2fd95d4 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:11.252838 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1804732-cbb8-436e-808f-8458f2fd95d4 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:11.254107 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1804732-cbb8-436e-808f-8458f2fd95d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:11.291025 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-b1804732-cbb8-436e-808f-8458f2fd95d4 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:15:11.292570 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-b1804732-cbb8-436e-808f-8458f2fd95d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:15:11.317235 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1110/2203] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:15:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:15:11.318411 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f3a270-4f92-477f-956d-6fad505724f3 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:11.330563 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f3a270-4f92-477f-956d-6fad505724f3 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:11.343162 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f74638c-c81e-4d63-83b4-316d153923bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:11.344539 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f74638c-c81e-4d63-83b4-316d153923bd admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:11.345244 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f74638c-c81e-4d63-83b4-316d153923bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:11.348690 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f74638c-c81e-4d63-83b4-316d153923bd admin admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:11.349665 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f74638c-c81e-4d63-83b4-316d153923bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:15:11.350165 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f74638c-c81e-4d63-83b4-316d153923bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:11.352313 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f74638c-c81e-4d63-83b4-316d153923bd admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:11.363974 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:15:11.379187 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1804732-cbb8-436e-808f-8458f2fd95d4 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:11.385970 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1094/2204] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:15:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:15:11.400557 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1804732-cbb8-436e-808f-8458f2fd95d4 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:11.409512 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489b7b2c-ce8d-4c99-96f8-f0cf28833f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:11.410228 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b7b2c-ce8d-4c99-96f8-f0cf28833f94 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:11.410756 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b7b2c-ce8d-4c99-96f8-f0cf28833f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:11.411152 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b7b2c-ce8d-4c99-96f8-f0cf28833f94 admin admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:11.411971 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489b7b2c-ce8d-4c99-96f8-f0cf28833f94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:11.414466 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489b7b2c-ce8d-4c99-96f8-f0cf28833f94 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:11.452716 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1111/2205] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:15:11 2022] POST /identity/v3/projects => generated 371 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:15:11.457264 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f74638c-c81e-4d63-83b4-316d153923bd admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:11.470765 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f74638c-c81e-4d63-83b4-316d153923bd admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:11.538696 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047b042c-4bd2-4664-a27b-26dde81be464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:11.539386 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b042c-4bd2-4664-a27b-26dde81be464 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:11.539767 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b042c-4bd2-4664-a27b-26dde81be464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:11.540190 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b042c-4bd2-4664-a27b-26dde81be464 admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:11.540928 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047b042c-4bd2-4664-a27b-26dde81be464 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:11.543016 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047b042c-4bd2-4664-a27b-26dde81be464 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:11.572211 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1095/2206] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:15:11 2022] POST /identity/v3/users => generated 286 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:15:11.576830 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489b7b2c-ce8d-4c99-96f8-f0cf28833f94 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:11.592207 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489b7b2c-ce8d-4c99-96f8-f0cf28833f94 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:11.601363 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e7b792-8f1e-4e89-9afd-d1588871efe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:11.602144 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e7b792-8f1e-4e89-9afd-d1588871efe2 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:11.602509 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e7b792-8f1e-4e89-9afd-d1588871efe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:11.603376 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e7b792-8f1e-4e89-9afd-d1588871efe2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:11.603786 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e7b792-8f1e-4e89-9afd-d1588871efe2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:11.607778 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e7b792-8f1e-4e89-9afd-d1588871efe2 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:11.646924 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1112/2207] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 01:15:11.650940 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047b042c-4bd2-4664-a27b-26dde81be464 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:11.662926 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047b042c-4bd2-4664-a27b-26dde81be464 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:11.673895 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f46127-ab15-480e-b9e3-a456a2424db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:11.674794 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f46127-ab15-480e-b9e3-a456a2424db4 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:11.675228 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f46127-ab15-480e-b9e3-a456a2424db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:11.675665 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f46127-ab15-480e-b9e3-a456a2424db4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:11.676528 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f46127-ab15-480e-b9e3-a456a2424db4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:11.678317 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f46127-ab15-480e-b9e3-a456a2424db4 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:11.691433 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1096/2208] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:15:11.695283 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e7b792-8f1e-4e89-9afd-d1588871efe2 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:11.717608 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e7b792-8f1e-4e89-9afd-d1588871efe2 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:11.723792 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b3def-efc3-45b2-9539-9a857894f182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:11.724617 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b3def-efc3-45b2-9539-9a857894f182 admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:11.724900 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b3def-efc3-45b2-9539-9a857894f182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:11.725184 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b3def-efc3-45b2-9539-9a857894f182 admin admin] PATH_INFO: `/v3/projects/8263d46be0db47ce9191c099759cf0e9/users/e5351a59a0294f59b6b26ddc22ba7ffa/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:11.745142 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8b3def-efc3-45b2-9539-9a857894f182 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8263d46be0db47ce9191c099759cf0e9, user_id=e5351a59a0294f59b6b26ddc22ba7ffa, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:11.748699 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8b3def-efc3-45b2-9539-9a857894f182 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:11.772516 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1113/2209] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:15:11 2022] PUT /identity/v3/projects/8263d46be0db47ce9191c099759cf0e9/users/e5351a59a0294f59b6b26ddc22ba7ffa/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:15:11.774638 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0464df47-f4c7-4f27-9dc8-9ca0bb9dcd3d None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:11.775097 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0464df47-f4c7-4f27-9dc8-9ca0bb9dcd3d None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:11.775502 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0464df47-f4c7-4f27-9dc8-9ca0bb9dcd3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:11.806132 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-0464df47-f4c7-4f27-9dc8-9ca0bb9dcd3d None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:15:11.807519 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-0464df47-f4c7-4f27-9dc8-9ca0bb9dcd3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:15:11.884418 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1097/2210] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:15:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:15:11.897768 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b3def-efc3-45b2-9539-9a857894f182 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:11.899905 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-cc8b3def-efc3-45b2-9539-9a857894f182 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:15:11.908122 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:15:11.991095 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b3def-efc3-45b2-9539-9a857894f182 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:11.995299 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2317d8-ba0a-4e83-be5b-430551751690 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:11.995966 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2317d8-ba0a-4e83-be5b-430551751690 service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:11.996292 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2317d8-ba0a-4e83-be5b-430551751690 service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:11.997731 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2317d8-ba0a-4e83-be5b-430551751690 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:12.008713 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2317d8-ba0a-4e83-be5b-430551751690 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:12.010704 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2317d8-ba0a-4e83-be5b-430551751690 service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:12.024261 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1114/2211] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:15:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:15:12.201954 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0464df47-f4c7-4f27-9dc8-9ca0bb9dcd3d None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:12.203461 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-0464df47-f4c7-4f27-9dc8-9ca0bb9dcd3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:15:12.244369 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0464df47-f4c7-4f27-9dc8-9ca0bb9dcd3d None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:12.271353 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8b928d-0ab2-4f1b-9174-bc83e64baf93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:12.272211 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b928d-0ab2-4f1b-9174-bc83e64baf93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:12.272640 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b928d-0ab2-4f1b-9174-bc83e64baf93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:12.273006 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b928d-0ab2-4f1b-9174-bc83e64baf93 admin admin] PATH_INFO: `/v3/users/c06f498366d54dba8a92f6c66d3bae0e` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:12.277739 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8b928d-0ab2-4f1b-9174-bc83e64baf93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c06f498366d54dba8a92f6c66d3bae0e)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:12.283691 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8b928d-0ab2-4f1b-9174-bc83e64baf93 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:12.417121 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-8d8b928d-0ab2-4f1b-9174-bc83e64baf93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c06f498366d54dba8a92f6c66d3bae0e'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:15:12.463565 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-8d8b928d-0ab2-4f1b-9174-bc83e64baf93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c06f498366d54dba8a92f6c66d3bae0e'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:15:12.484222 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-8d8b928d-0ab2-4f1b-9174-bc83e64baf93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c06f498366d54dba8a92f6c66d3bae0e'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:15:12.484919 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-8d8b928d-0ab2-4f1b-9174-bc83e64baf93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c06f498366d54dba8a92f6c66d3bae0e'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:15:12.487642 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:15:12.499861 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-8d8b928d-0ab2-4f1b-9174-bc83e64baf93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c06f498366d54dba8a92f6c66d3bae0e'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:15:12.514351 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1098/2212] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:15:12 2022] DELETE /identity/v3/users/c06f498366d54dba8a92f6c66d3bae0e => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:15:12.517386 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b0cd8-62fd-49b8-9d70-9dee310b25e5 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:12.517386 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b0cd8-62fd-49b8-9d70-9dee310b25e5 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:12.517386 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b0cd8-62fd-49b8-9d70-9dee310b25e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:12.556501 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-218b0cd8-62fd-49b8-9d70-9dee310b25e5 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:15:12.558928 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-218b0cd8-62fd-49b8-9d70-9dee310b25e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:15:12.716616 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1115/2213] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:15:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:15:12.726799 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8b928d-0ab2-4f1b-9174-bc83e64baf93 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:12.729366 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-8d8b928d-0ab2-4f1b-9174-bc83e64baf93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:15:12.941132 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8b928d-0ab2-4f1b-9174-bc83e64baf93 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:12.951295 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb721ba-bd75-4151-9eca-36e3728b3b2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:12.954085 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb721ba-bd75-4151-9eca-36e3728b3b2e service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:12.954377 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb721ba-bd75-4151-9eca-36e3728b3b2e service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:12.954654 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb721ba-bd75-4151-9eca-36e3728b3b2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:12.975123 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb721ba-bd75-4151-9eca-36e3728b3b2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:12.977430 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb721ba-bd75-4151-9eca-36e3728b3b2e service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:12.995843 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1099/2214] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:15:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:15:13.791608 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218b0cd8-62fd-49b8-9d70-9dee310b25e5 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:13.793199 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-218b0cd8-62fd-49b8-9d70-9dee310b25e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:15:13.818044 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218b0cd8-62fd-49b8-9d70-9dee310b25e5 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:13.835273 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4681d0-d0dd-4427-87dd-f287e569deb7 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:13.835862 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4681d0-d0dd-4427-87dd-f287e569deb7 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:13.835862 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4681d0-d0dd-4427-87dd-f287e569deb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:13.845206 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5a03ee-b29f-4173-b6e6-326aa0cc1be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:13.846281 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5a03ee-b29f-4173-b6e6-326aa0cc1be4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:13.846780 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5a03ee-b29f-4173-b6e6-326aa0cc1be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:13.847210 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5a03ee-b29f-4173-b6e6-326aa0cc1be4 admin admin] PATH_INFO: `/v3/projects/6b2a99457ced46a5a4411d35efbe736e` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:13.849879 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5a03ee-b29f-4173-b6e6-326aa0cc1be4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b2a99457ced46a5a4411d35efbe736e)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:13.855664 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5a03ee-b29f-4173-b6e6-326aa0cc1be4 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:13.893222 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-ae4681d0-d0dd-4427-87dd-f287e569deb7 None None] MFA Rules not processed for user `e5351a59a0294f59b6b26ddc22ba7ffa`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:15:13.894478 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-ae4681d0-d0dd-4427-87dd-f287e569deb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:15:14.005462 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-ff5a03ee-b29f-4173-b6e6-326aa0cc1be4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b2a99457ced46a5a4411d35efbe736e', 'request_id': 'req-ff5a03ee-b29f-4173-b6e6-326aa0cc1be4'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:15:14.057104 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-ff5a03ee-b29f-4173-b6e6-326aa0cc1be4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b2a99457ced46a5a4411d35efbe736e', 'request_id': 'req-ff5a03ee-b29f-4173-b6e6-326aa0cc1be4'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:15:14.091907 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-ff5a03ee-b29f-4173-b6e6-326aa0cc1be4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b2a99457ced46a5a4411d35efbe736e', 'request_id': 'req-ff5a03ee-b29f-4173-b6e6-326aa0cc1be4'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:15:14.173119 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-ff5a03ee-b29f-4173-b6e6-326aa0cc1be4 admin admin] The token cache is being invalidate because project 6b2a99457ced46a5a4411d35efbe736e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:15:14.175638 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-ff5a03ee-b29f-4173-b6e6-326aa0cc1be4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:15:14.179919 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-ff5a03ee-b29f-4173-b6e6-326aa0cc1be4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:15:14.193501 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1116/2215] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:15:13 2022] DELETE /identity/v3/projects/6b2a99457ced46a5a4411d35efbe736e => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:15:14.196196 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5a03ee-b29f-4173-b6e6-326aa0cc1be4 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:14.197355 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1100/2216] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:15:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:15:14.201304 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d36acf7-2f1d-4af4-89ca-f367861852e1 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:14.205387 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d36acf7-2f1d-4af4-89ca-f367861852e1 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:14.205903 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d36acf7-2f1d-4af4-89ca-f367861852e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:14.209537 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-ff5a03ee-b29f-4173-b6e6-326aa0cc1be4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:15:14.281821 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-4d36acf7-2f1d-4af4-89ca-f367861852e1 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:15:14.284032 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-4d36acf7-2f1d-4af4-89ca-f367861852e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:15:14.369087 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:15:14.441666 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5a03ee-b29f-4173-b6e6-326aa0cc1be4 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:14.451024 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6071ac5b-6ac0-4a50-9286-6a5758600e63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:14.453194 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6071ac5b-6ac0-4a50-9286-6a5758600e63 service cinder] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:14.454593 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6071ac5b-6ac0-4a50-9286-6a5758600e63 service cinder] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:14.455331 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6071ac5b-6ac0-4a50-9286-6a5758600e63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:14.460266 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-6071ac5b-6ac0-4a50-9286-6a5758600e63 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:15:14.470108 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1101/2217] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:15:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:15:14.478662 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb8eaf7-07c5-4443-ad88-2acd9e0b1706 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:14.479001 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb8eaf7-07c5-4443-ad88-2acd9e0b1706 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:14.479471 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb8eaf7-07c5-4443-ad88-2acd9e0b1706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:14.526863 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-ffb8eaf7-07c5-4443-ad88-2acd9e0b1706 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:15:14.529785 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-ffb8eaf7-07c5-4443-ad88-2acd9e0b1706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:15:14.555219 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1102/2218] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:15:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:15:14.561219 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb8eaf7-07c5-4443-ad88-2acd9e0b1706 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:14.579587 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb8eaf7-07c5-4443-ad88-2acd9e0b1706 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:14.591564 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd8469c-6305-405a-8b8d-5565721912ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:14.592198 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd8469c-6305-405a-8b8d-5565721912ad admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:14.592479 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd8469c-6305-405a-8b8d-5565721912ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:14.592752 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd8469c-6305-405a-8b8d-5565721912ad admin admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:14.593413 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd8469c-6305-405a-8b8d-5565721912ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:15:14.593715 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd8469c-6305-405a-8b8d-5565721912ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:14.595352 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd8469c-6305-405a-8b8d-5565721912ad admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:14.615509 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6071ac5b-6ac0-4a50-9286-6a5758600e63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:14.623892 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6071ac5b-6ac0-4a50-9286-6a5758600e63 service cinder] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:14.650095 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1103/2219] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:15:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:15:14.651166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd8469c-6305-405a-8b8d-5565721912ad admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:14.657153 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1117/2220] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:15:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:15:14.661562 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:15:14.673693 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd8469c-6305-405a-8b8d-5565721912ad admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:14.690257 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de387f5-69ac-480e-a1e8-6de9f1183a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:14.691304 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de387f5-69ac-480e-a1e8-6de9f1183a67 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:14.692452 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de387f5-69ac-480e-a1e8-6de9f1183a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:14.692733 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de387f5-69ac-480e-a1e8-6de9f1183a67 admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:14.693579 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de387f5-69ac-480e-a1e8-6de9f1183a67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:14.695961 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de387f5-69ac-480e-a1e8-6de9f1183a67 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:14.807860 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1104/2221] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:15:14 2022] POST /identity/v3/projects => generated 373 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:15:14.810479 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6071ac5b-6ac0-4a50-9286-6a5758600e63 service cinder] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:14.835289 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6071ac5b-6ac0-4a50-9286-6a5758600e63 service cinder] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:15.017455 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f257cc-025e-450d-bf30-eb23c4fd64c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:15.018091 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f257cc-025e-450d-bf30-eb23c4fd64c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:15.018375 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f257cc-025e-450d-bf30-eb23c4fd64c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:15.019925 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f257cc-025e-450d-bf30-eb23c4fd64c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:15.019925 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f257cc-025e-450d-bf30-eb23c4fd64c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:15.021261 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f257cc-025e-450d-bf30-eb23c4fd64c8 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:15.070548 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1118/2222] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:15:14 2022] POST /identity/v3/users => generated 287 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:15:15.075409 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de387f5-69ac-480e-a1e8-6de9f1183a67 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:15.109162 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de387f5-69ac-480e-a1e8-6de9f1183a67 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:15.128629 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6145dfa1-8634-4f63-8150-149c2581b675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:15.129715 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6145dfa1-8634-4f63-8150-149c2581b675 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:15.134739 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6145dfa1-8634-4f63-8150-149c2581b675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:15.135594 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6145dfa1-8634-4f63-8150-149c2581b675 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:15.137455 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6145dfa1-8634-4f63-8150-149c2581b675 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:15.140174 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6145dfa1-8634-4f63-8150-149c2581b675 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:15.208972 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1105/2223] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:15:15 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:15:15.213092 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f257cc-025e-450d-bf30-eb23c4fd64c8 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:15.230809 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f257cc-025e-450d-bf30-eb23c4fd64c8 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:15.260870 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c19d74c-e479-4cce-aba2-c00a949f58d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:15.261511 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19d74c-e479-4cce-aba2-c00a949f58d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:15.261791 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19d74c-e479-4cce-aba2-c00a949f58d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:15.262065 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19d74c-e479-4cce-aba2-c00a949f58d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:15.262775 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c19d74c-e479-4cce-aba2-c00a949f58d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:15.264705 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c19d74c-e479-4cce-aba2-c00a949f58d7 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:15.300094 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1119/2224] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:15:15 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:15:15.304370 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6145dfa1-8634-4f63-8150-149c2581b675 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:15.334511 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6145dfa1-8634-4f63-8150-149c2581b675 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:15.344084 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccff3fa-116a-4e38-9712-1bc320ad8c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:15.344802 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccff3fa-116a-4e38-9712-1bc320ad8c57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:15.345086 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccff3fa-116a-4e38-9712-1bc320ad8c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:15.345367 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccff3fa-116a-4e38-9712-1bc320ad8c57 admin admin] PATH_INFO: `/v3/projects/5deaa920a59448268a4151c845bb7851/users/9851332c4fdf4874bb3e860f0035fd4b/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:15.379642 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccff3fa-116a-4e38-9712-1bc320ad8c57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5deaa920a59448268a4151c845bb7851, user_id=9851332c4fdf4874bb3e860f0035fd4b, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:15.382389 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccff3fa-116a-4e38-9712-1bc320ad8c57 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:15.424412 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1106/2225] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:15:15 2022] PUT /identity/v3/projects/5deaa920a59448268a4151c845bb7851/users/9851332c4fdf4874bb3e860f0035fd4b/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:15:15.428728 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8902fc-a741-4131-8eb6-c3e229167d83 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:15.429016 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8902fc-a741-4131-8eb6-c3e229167d83 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:15.429291 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8902fc-a741-4131-8eb6-c3e229167d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:15.482522 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-3e8902fc-a741-4131-8eb6-c3e229167d83 None None] MFA Rules not processed for user `9851332c4fdf4874bb3e860f0035fd4b`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:15:15.484233 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-3e8902fc-a741-4131-8eb6-c3e229167d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:15:15.567919 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:15:15.652096 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1120/2226] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:15:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:15:15.661217 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccff3fa-116a-4e38-9712-1bc320ad8c57 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:15.773133 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccff3fa-116a-4e38-9712-1bc320ad8c57 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:15.780411 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24255d6-18b8-438a-b501-ee93c50c61df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:15.781132 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24255d6-18b8-438a-b501-ee93c50c61df service cinder] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:15.781637 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24255d6-18b8-438a-b501-ee93c50c61df service cinder] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:15.782273 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24255d6-18b8-438a-b501-ee93c50c61df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:15.805187 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24255d6-18b8-438a-b501-ee93c50c61df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:15.807996 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24255d6-18b8-438a-b501-ee93c50c61df service cinder] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:15.822668 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1107/2227] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:15:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:15:16.510352 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd49abc-1cb0-4412-8842-49562a1c029c None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:16.510807 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd49abc-1cb0-4412-8842-49562a1c029c None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:16.510883 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd49abc-1cb0-4412-8842-49562a1c029c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:16.585000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-7fd49abc-1cb0-4412-8842-49562a1c029c None None] MFA Rules not processed for user `e5351a59a0294f59b6b26ddc22ba7ffa`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:15:16.604927 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-7fd49abc-1cb0-4412-8842-49562a1c029c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:15:16.779833 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1121/2228] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:15:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:15:17.669948 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8cfe0d-c32a-4039-84cb-edd316e41750 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:17.670803 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8cfe0d-c32a-4039-84cb-edd316e41750 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:17.671138 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8cfe0d-c32a-4039-84cb-edd316e41750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:17.714950 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-4c8cfe0d-c32a-4039-84cb-edd316e41750 None None] MFA Rules not processed for user `9851332c4fdf4874bb3e860f0035fd4b`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:15:17.717052 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-4c8cfe0d-c32a-4039-84cb-edd316e41750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:15:17.745074 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1108/2229] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:15:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:15:23.315508 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09202b1-95ce-401f-8fd1-c617e242a0db None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:23.316705 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09202b1-95ce-401f-8fd1-c617e242a0db None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:23.317270 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09202b1-95ce-401f-8fd1-c617e242a0db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:23.339732 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-e09202b1-95ce-401f-8fd1-c617e242a0db None None] MFA Rules not processed for user `5411bb3267704943aea31c4bf1b4c9d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:15:23.341630 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-e09202b1-95ce-401f-8fd1-c617e242a0db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:15:23.450778 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1122/2230] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:15:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:15:24.295459 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc4758-2406-497c-8052-031ebe1efc61 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:24.296607 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc4758-2406-497c-8052-031ebe1efc61 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:24.296954 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc4758-2406-497c-8052-031ebe1efc61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:24.318520 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-a4cc4758-2406-497c-8052-031ebe1efc61 None None] MFA Rules not processed for user `20af4425b7434d2dab7baeacbb845387`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:15:24.319855 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-a4cc4758-2406-497c-8052-031ebe1efc61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:15:24.390261 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1109/2231] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:15:24 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:15:28.581521 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02632c5-d6f3-4395-bad3-7d4c857a3e74 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:28.582058 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02632c5-d6f3-4395-bad3-7d4c857a3e74 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:28.582058 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02632c5-d6f3-4395-bad3-7d4c857a3e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:28.615747 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-b02632c5-d6f3-4395-bad3-7d4c857a3e74 None None] MFA Rules not processed for user `de75fd7d7721433684cafa2e6d3faaae`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:15:28.617357 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-b02632c5-d6f3-4395-bad3-7d4c857a3e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:15:28.710620 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1123/2232] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Sep 8 01:15:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:15:28.724538 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cc4758-2406-497c-8052-031ebe1efc61 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:28.726292 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-a4cc4758-2406-497c-8052-031ebe1efc61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:15:28.826836 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:15:28.853174 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cc4758-2406-497c-8052-031ebe1efc61 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:28.860321 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f01ad6-cba9-4ebb-8d89-69b82c24a3e8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8add26872fe6479aa9d7de155a17195a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:28.861026 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f01ad6-cba9-4ebb-8d89-69b82c24a3e8 service swift] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:28.861377 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f01ad6-cba9-4ebb-8d89-69b82c24a3e8 service swift] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:28.861722 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f01ad6-cba9-4ebb-8d89-69b82c24a3e8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:28.870962 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f01ad6-cba9-4ebb-8d89-69b82c24a3e8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:28.872812 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f01ad6-cba9-4ebb-8d89-69b82c24a3e8 service swift] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:28.882162 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1110/2233] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Sep 8 01:15:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 01:15:38.699579 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=79 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:15:38.878932 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=10 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:15:48.806361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331832e8-7076-45c0-88ac-a3901066aa77 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:48.807216 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331832e8-7076-45c0-88ac-a3901066aa77 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:48.807501 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331832e8-7076-45c0-88ac-a3901066aa77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:48.828938 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-331832e8-7076-45c0-88ac-a3901066aa77 None None] MFA Rules not processed for user `de75fd7d7721433684cafa2e6d3faaae`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:15:48.830606 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-331832e8-7076-45c0-88ac-a3901066aa77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:15:48.846485 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1124/2234] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Sep 8 01:15:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:15:48.861858 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f01ad6-cba9-4ebb-8d89-69b82c24a3e8 service swift] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:15:48.899900 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f01ad6-cba9-4ebb-8d89-69b82c24a3e8 service swift] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:15:48.916861 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f686ca-4614-4f7c-a42a-0c659280cb10 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8add26872fe6479aa9d7de155a17195a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:15:48.918291 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f686ca-4614-4f7c-a42a-0c659280cb10 service swift] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:48.919217 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f686ca-4614-4f7c-a42a-0c659280cb10 service swift] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:48.920005 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f686ca-4614-4f7c-a42a-0c659280cb10 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:48.933382 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f686ca-4614-4f7c-a42a-0c659280cb10 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:15:48.935191 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f686ca-4614-4f7c-a42a-0c659280cb10 service swift] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:15:48.952841 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1111/2235] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Sep 8 01:15:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 01:15:53.016447 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0bad0f-d64c-4e90-a609-41a4978e5a22 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:15:53.017484 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0bad0f-d64c-4e90-a609-41a4978e5a22 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:15:53.018119 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0bad0f-d64c-4e90-a609-41a4978e5a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:15:53.042973 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-dc0bad0f-d64c-4e90-a609-41a4978e5a22 None None] MFA Rules not processed for user `e5351a59a0294f59b6b26ddc22ba7ffa`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:15:53.044935 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-dc0bad0f-d64c-4e90-a609-41a4978e5a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:15:53.056749 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1125/2236] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:15:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:15:58.944577 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=6 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:16:02.696828 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb7065-7f94-4d78-89d0-fa277b632994 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:16:02.697873 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb7065-7f94-4d78-89d0-fa277b632994 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:16:02.698529 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb7065-7f94-4d78-89d0-fa277b632994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:16:02.723352 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-fdeb7065-7f94-4d78-89d0-fa277b632994 None None] MFA Rules not processed for user `de75fd7d7721433684cafa2e6d3faaae`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:16:02.724594 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-fdeb7065-7f94-4d78-89d0-fa277b632994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:16:02.733918 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1112/2237] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Sep 8 01:16:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:16:02.750102 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0bad0f-d64c-4e90-a609-41a4978e5a22 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:16:02.764659 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0bad0f-d64c-4e90-a609-41a4978e5a22 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:16:02.771672 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6843428-1109-484b-b411-2506b4a25392 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8add26872fe6479aa9d7de155a17195a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:16:02.772430 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6843428-1109-484b-b411-2506b4a25392 service swift] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:16:02.772875 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6843428-1109-484b-b411-2506b4a25392 service swift] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:16:02.773336 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6843428-1109-484b-b411-2506b4a25392 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:16:02.786753 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6843428-1109-484b-b411-2506b4a25392 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:16:02.788812 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6843428-1109-484b-b411-2506b4a25392 service swift] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:16:02.800248 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1126/2238] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Sep 8 01:16:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 01:16:03.406869 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814d879-9354-4c93-b524-650abc267f7b None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:16:03.406869 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814d879-9354-4c93-b524-650abc267f7b None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:16:03.407188 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814d879-9354-4c93-b524-650abc267f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:16:03.430503 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-9814d879-9354-4c93-b524-650abc267f7b None None] MFA Rules not processed for user `de75fd7d7721433684cafa2e6d3faaae`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:16:03.431729 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-9814d879-9354-4c93-b524-650abc267f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:16:03.440898 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1113/2239] 10.222.0.42 () {58 vars in 967 bytes} [Thu Sep 8 01:16:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:16:03.454965 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6843428-1109-484b-b411-2506b4a25392 service swift] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:16:03.465962 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6843428-1109-484b-b411-2506b4a25392 service swift] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:16:03.471240 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b023caf7-6d93-4b26-8dca-ab5b1a9732ef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8add26872fe6479aa9d7de155a17195a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:16:03.471839 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b023caf7-6d93-4b26-8dca-ab5b1a9732ef service swift] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:16:03.472116 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b023caf7-6d93-4b26-8dca-ab5b1a9732ef service swift] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:16:03.472415 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b023caf7-6d93-4b26-8dca-ab5b1a9732ef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:16:03.481099 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b023caf7-6d93-4b26-8dca-ab5b1a9732ef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:16:03.482708 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b023caf7-6d93-4b26-8dca-ab5b1a9732ef service swift] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:16:03.491138 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1127/2240] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Sep 8 01:16:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 01:16:03.788483 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d64bab-b262-4948-b5d2-d35ca9bb910b None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:16:03.788985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d64bab-b262-4948-b5d2-d35ca9bb910b None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:16:03.789077 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d64bab-b262-4948-b5d2-d35ca9bb910b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:16:03.809635 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-d0d64bab-b262-4948-b5d2-d35ca9bb910b None None] MFA Rules not processed for user `de75fd7d7721433684cafa2e6d3faaae`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:16:03.810937 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-d0d64bab-b262-4948-b5d2-d35ca9bb910b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:16:03.818173 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1114/2241] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Sep 8 01:16:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:16:03.830368 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b023caf7-6d93-4b26-8dca-ab5b1a9732ef service swift] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:16:03.841158 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b023caf7-6d93-4b26-8dca-ab5b1a9732ef service swift] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:16:03.846350 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4050255e-2a93-487f-af42-44761cb14568 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8add26872fe6479aa9d7de155a17195a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:16:03.846948 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050255e-2a93-487f-af42-44761cb14568 service swift] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:16:03.847226 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050255e-2a93-487f-af42-44761cb14568 service swift] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:16:03.847523 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050255e-2a93-487f-af42-44761cb14568 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:16:03.857049 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4050255e-2a93-487f-af42-44761cb14568 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:16:03.858770 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4050255e-2a93-487f-af42-44761cb14568 service swift] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:16:03.867326 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1128/2242] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Sep 8 01:16:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 01:16:04.207851 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc14e171-d3ae-465b-aa5d-768857dd01a8 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:16:04.208171 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc14e171-d3ae-465b-aa5d-768857dd01a8 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:16:04.208653 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc14e171-d3ae-465b-aa5d-768857dd01a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:16:04.229098 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-cc14e171-d3ae-465b-aa5d-768857dd01a8 None None] MFA Rules not processed for user `de75fd7d7721433684cafa2e6d3faaae`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:16:04.230916 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-cc14e171-d3ae-465b-aa5d-768857dd01a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:16:04.238542 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1115/2243] 10.222.0.42 () {58 vars in 967 bytes} [Thu Sep 8 01:16:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:16:04.255810 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4050255e-2a93-487f-af42-44761cb14568 service swift] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:16:04.267826 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4050255e-2a93-487f-af42-44761cb14568 service swift] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:16:04.273811 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b10154f-bf71-4079-913f-9756032822ec service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8add26872fe6479aa9d7de155a17195a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:16:04.274621 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b10154f-bf71-4079-913f-9756032822ec service swift] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:16:04.274906 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b10154f-bf71-4079-913f-9756032822ec service swift] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:16:04.275192 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b10154f-bf71-4079-913f-9756032822ec service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:16:04.285008 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b10154f-bf71-4079-913f-9756032822ec service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:16:04.286831 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b10154f-bf71-4079-913f-9756032822ec service swift] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:16:04.297825 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1129/2244] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Sep 8 01:16:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 01:16:06.524928 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc14e171-d3ae-465b-aa5d-768857dd01a8 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:16:06.590971 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc14e171-d3ae-465b-aa5d-768857dd01a8 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:16:06.596308 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41355b6-3708-4022-9956-cb128aca217a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:16:06.596941 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41355b6-3708-4022-9956-cb128aca217a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:16:06.597218 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41355b6-3708-4022-9956-cb128aca217a admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:16:06.597493 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41355b6-3708-4022-9956-cb128aca217a admin admin] PATH_INFO: `/v3/users/9851332c4fdf4874bb3e860f0035fd4b` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:16:06.598644 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41355b6-3708-4022-9956-cb128aca217a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9851332c4fdf4874bb3e860f0035fd4b)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:16:06.600677 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41355b6-3708-4022-9956-cb128aca217a admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:16:06.652052 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-e41355b6-3708-4022-9956-cb128aca217a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9851332c4fdf4874bb3e860f0035fd4b'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:16:06.667375 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-e41355b6-3708-4022-9956-cb128aca217a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9851332c4fdf4874bb3e860f0035fd4b'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:16:06.674967 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-e41355b6-3708-4022-9956-cb128aca217a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9851332c4fdf4874bb3e860f0035fd4b'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:16:06.675678 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-e41355b6-3708-4022-9956-cb128aca217a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9851332c4fdf4874bb3e860f0035fd4b'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:16:06.683630 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-e41355b6-3708-4022-9956-cb128aca217a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9851332c4fdf4874bb3e860f0035fd4b'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:16:06.689953 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1116/2245] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:16:06 2022] DELETE /identity/v3/users/9851332c4fdf4874bb3e860f0035fd4b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:16:06.694619 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609cabbf-8961-4fa6-a82a-303ca960687c None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:16:06.695071 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609cabbf-8961-4fa6-a82a-303ca960687c None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:16:06.695186 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609cabbf-8961-4fa6-a82a-303ca960687c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:16:06.713955 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-609cabbf-8961-4fa6-a82a-303ca960687c None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:16:06.715562 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-609cabbf-8961-4fa6-a82a-303ca960687c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:16:06.778568 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1130/2246] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:16:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:16:06.788870 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41355b6-3708-4022-9956-cb128aca217a admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:16:06.791032 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-e41355b6-3708-4022-9956-cb128aca217a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:16:06.850356 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41355b6-3708-4022-9956-cb128aca217a admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:16:06.854044 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dbec44-d942-4667-898a-7f6053760edb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:16:06.854657 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dbec44-d942-4667-898a-7f6053760edb service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:16:06.854942 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dbec44-d942-4667-898a-7f6053760edb service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:16:06.855217 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dbec44-d942-4667-898a-7f6053760edb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:16:06.863318 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:16:06.865581 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dbec44-d942-4667-898a-7f6053760edb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:16:06.867334 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dbec44-d942-4667-898a-7f6053760edb service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:16:06.880900 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1117/2247] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:16:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:16:07.222810 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609cabbf-8961-4fa6-a82a-303ca960687c None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:16:07.224498 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-609cabbf-8961-4fa6-a82a-303ca960687c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:16:07.242872 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609cabbf-8961-4fa6-a82a-303ca960687c None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:16:07.250785 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7ed350-136d-4c0e-ae89-9d3afc300997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:16:07.251769 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7ed350-136d-4c0e-ae89-9d3afc300997 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:16:07.252090 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7ed350-136d-4c0e-ae89-9d3afc300997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:16:07.252375 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7ed350-136d-4c0e-ae89-9d3afc300997 admin admin] PATH_INFO: `/v3/projects/5deaa920a59448268a4151c845bb7851` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:16:07.253612 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7ed350-136d-4c0e-ae89-9d3afc300997 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5deaa920a59448268a4151c845bb7851)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:16:07.255957 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7ed350-136d-4c0e-ae89-9d3afc300997 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:16:07.317969 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-fc7ed350-136d-4c0e-ae89-9d3afc300997 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5deaa920a59448268a4151c845bb7851', 'request_id': 'req-fc7ed350-136d-4c0e-ae89-9d3afc300997'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:16:07.333038 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-fc7ed350-136d-4c0e-ae89-9d3afc300997 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5deaa920a59448268a4151c845bb7851', 'request_id': 'req-fc7ed350-136d-4c0e-ae89-9d3afc300997'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:16:07.342889 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-fc7ed350-136d-4c0e-ae89-9d3afc300997 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5deaa920a59448268a4151c845bb7851', 'request_id': 'req-fc7ed350-136d-4c0e-ae89-9d3afc300997'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:16:07.358381 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-fc7ed350-136d-4c0e-ae89-9d3afc300997 admin admin] The token cache is being invalidate because project 5deaa920a59448268a4151c845bb7851 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:16:07.358741 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-fc7ed350-136d-4c0e-ae89-9d3afc300997 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:16:07.360297 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-fc7ed350-136d-4c0e-ae89-9d3afc300997 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:16:07.362228 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1131/2248] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:16:07 2022] DELETE /identity/v3/projects/5deaa920a59448268a4151c845bb7851 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:16:07.368227 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb8deec-50d9-4706-b632-db2379e69172 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:16:07.368489 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb8deec-50d9-4706-b632-db2379e69172 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:16:07.369260 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb8deec-50d9-4706-b632-db2379e69172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:16:07.396941 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-6bb8deec-50d9-4706-b632-db2379e69172 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:16:07.398085 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-6bb8deec-50d9-4706-b632-db2379e69172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:16:07.465416 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1118/2249] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:16:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:16:07.472944 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c5e29-8f54-4190-875d-9928fc45d506 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:16:07.473228 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c5e29-8f54-4190-875d-9928fc45d506 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:16:07.473502 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c5e29-8f54-4190-875d-9928fc45d506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:16:07.493835 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-bc3c5e29-8f54-4190-875d-9928fc45d506 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:16:07.495048 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-bc3c5e29-8f54-4190-875d-9928fc45d506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:16:07.517272 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1132/2250] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:16:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:16:07.521688 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb8deec-50d9-4706-b632-db2379e69172 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:16:07.532530 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb8deec-50d9-4706-b632-db2379e69172 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:16:07.540265 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a80251b-ca12-4545-89db-c9afb7e06c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:16:07.542103 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80251b-ca12-4545-89db-c9afb7e06c46 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:16:07.542460 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80251b-ca12-4545-89db-c9afb7e06c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:16:07.542800 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80251b-ca12-4545-89db-c9afb7e06c46 admin admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:16:07.543811 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a80251b-ca12-4545-89db-c9afb7e06c46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:16:07.543929 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a80251b-ca12-4545-89db-c9afb7e06c46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:16:07.546412 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a80251b-ca12-4545-89db-c9afb7e06c46 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:16:07.562473 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1119/2251] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:16:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:16:07.567242 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3c5e29-8f54-4190-875d-9928fc45d506 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:16:07.578742 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3c5e29-8f54-4190-875d-9928fc45d506 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:16:07.588790 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297adb43-7687-4cf6-9cfa-527d6df46093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:16:07.589452 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297adb43-7687-4cf6-9cfa-527d6df46093 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:16:07.589733 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297adb43-7687-4cf6-9cfa-527d6df46093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:16:07.590009 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297adb43-7687-4cf6-9cfa-527d6df46093 admin admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:16:07.590675 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297adb43-7687-4cf6-9cfa-527d6df46093 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:16:07.592717 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297adb43-7687-4cf6-9cfa-527d6df46093 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:16:07.603132 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:16:07.633764 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1133/2252] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:16:07 2022] POST /identity/v3/projects => generated 383 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 01:16:07.639022 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a80251b-ca12-4545-89db-c9afb7e06c46 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:16:07.652094 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a80251b-ca12-4545-89db-c9afb7e06c46 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:16:07.717861 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e9504b-c885-47ae-8347-880a1bbb7d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:16:07.718554 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e9504b-c885-47ae-8347-880a1bbb7d95 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:16:07.718802 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e9504b-c885-47ae-8347-880a1bbb7d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:16:07.719120 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e9504b-c885-47ae-8347-880a1bbb7d95 admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:16:07.719805 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e9504b-c885-47ae-8347-880a1bbb7d95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:16:07.722149 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e9504b-c885-47ae-8347-880a1bbb7d95 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:16:07.744397 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1120/2253] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:16:07 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:16:07.750774 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297adb43-7687-4cf6-9cfa-527d6df46093 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:16:07.763444 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297adb43-7687-4cf6-9cfa-527d6df46093 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:16:07.769125 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdac92cf-9bc2-4994-a932-66bc9ef6d9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:16:07.769741 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac92cf-9bc2-4994-a932-66bc9ef6d9b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:16:07.770022 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac92cf-9bc2-4994-a932-66bc9ef6d9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:16:07.770296 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac92cf-9bc2-4994-a932-66bc9ef6d9b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:16:07.770945 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdac92cf-9bc2-4994-a932-66bc9ef6d9b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:16:07.772755 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdac92cf-9bc2-4994-a932-66bc9ef6d9b2 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:16:07.784246 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1134/2254] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:16:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:16:07.789196 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e9504b-c885-47ae-8347-880a1bbb7d95 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:16:07.801743 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e9504b-c885-47ae-8347-880a1bbb7d95 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:16:07.807291 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e6f3bd-7903-46ba-bb97-d2635dcc6d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:16:07.807971 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e6f3bd-7903-46ba-bb97-d2635dcc6d84 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:16:07.808259 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e6f3bd-7903-46ba-bb97-d2635dcc6d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:16:07.808530 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e6f3bd-7903-46ba-bb97-d2635dcc6d84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:16:07.809200 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e6f3bd-7903-46ba-bb97-d2635dcc6d84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:16:07.811887 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e6f3bd-7903-46ba-bb97-d2635dcc6d84 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:16:07.827273 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1121/2255] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:16:07 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:16:07.831274 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdac92cf-9bc2-4994-a932-66bc9ef6d9b2 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:16:07.843152 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdac92cf-9bc2-4994-a932-66bc9ef6d9b2 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:16:07.849584 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbc1637-5300-4a15-87d5-2603d1a5a154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:16:07.850767 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc1637-5300-4a15-87d5-2603d1a5a154 admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:16:07.851243 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc1637-5300-4a15-87d5-2603d1a5a154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:16:07.851667 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc1637-5300-4a15-87d5-2603d1a5a154 admin admin] PATH_INFO: `/v3/projects/93477e5edb38498ba32feb52519ba584/users/336e7f07cd7d461c8257995a772ac1f0/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:16:07.870777 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbc1637-5300-4a15-87d5-2603d1a5a154 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93477e5edb38498ba32feb52519ba584, user_id=336e7f07cd7d461c8257995a772ac1f0, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:16:07.874157 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbc1637-5300-4a15-87d5-2603d1a5a154 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:16:07.896359 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1135/2256] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:16:07 2022] PUT /identity/v3/projects/93477e5edb38498ba32feb52519ba584/users/336e7f07cd7d461c8257995a772ac1f0/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:16:07.905538 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d9397-eac1-416e-98fa-44ce1237c893 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:16:07.906212 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d9397-eac1-416e-98fa-44ce1237c893 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:16:07.906777 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d9397-eac1-416e-98fa-44ce1237c893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:16:07.933777 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-914d9397-eac1-416e-98fa-44ce1237c893 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:16:07.935950 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-914d9397-eac1-416e-98fa-44ce1237c893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:16:08.007747 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1122/2257] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:16:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:16:08.017523 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbc1637-5300-4a15-87d5-2603d1a5a154 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:16:08.019217 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-abbc1637-5300-4a15-87d5-2603d1a5a154 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:16:08.078602 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbc1637-5300-4a15-87d5-2603d1a5a154 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:16:08.082792 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a82f8d6-a5da-4c1f-b39f-c6edf94c3fac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:16:08.083755 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a82f8d6-a5da-4c1f-b39f-c6edf94c3fac service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:16:08.083755 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a82f8d6-a5da-4c1f-b39f-c6edf94c3fac service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:16:08.084256 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a82f8d6-a5da-4c1f-b39f-c6edf94c3fac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:16:08.097019 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a82f8d6-a5da-4c1f-b39f-c6edf94c3fac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:16:08.098748 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a82f8d6-a5da-4c1f-b39f-c6edf94c3fac service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:16:08.109858 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1136/2258] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:16:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:16:08.179057 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c23eea-956b-4dff-87f7-b88da3ed0f3b None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:16:08.179398 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c23eea-956b-4dff-87f7-b88da3ed0f3b None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:16:08.179675 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c23eea-956b-4dff-87f7-b88da3ed0f3b None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:16:08.181456 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1123/2259] 10.222.0.42 () {54 vars in 921 bytes} [Thu Sep 8 01:16:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:16:08.193534 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bbccac-ba34-428b-97a1-debf4dd10924 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:16:08.193823 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bbccac-ba34-428b-97a1-debf4dd10924 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:16:08.194106 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bbccac-ba34-428b-97a1-debf4dd10924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:16:08.219959 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-00bbccac-ba34-428b-97a1-debf4dd10924 None None] MFA Rules not processed for user `ae643cfdcb164aa9b474747cd21aa116`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:16:08.221423 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-00bbccac-ba34-428b-97a1-debf4dd10924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:16:08.322624 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1137/2260] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Sep 8 01:16:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:16:08.345583 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c23eea-956b-4dff-87f7-b88da3ed0f3b None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:16:08.348105 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-c4c23eea-956b-4dff-87f7-b88da3ed0f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:16:08.358917 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:16:08.452857 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c23eea-956b-4dff-87f7-b88da3ed0f3b None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:16:08.457228 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaa60f3-673b-4a21-a750-d49fd275ac12 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1acfa7a09d34e2a9045c3f3fbea0f52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:16:08.458053 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa60f3-673b-4a21-a750-d49fd275ac12 service glance] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:16:08.458492 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa60f3-673b-4a21-a750-d49fd275ac12 service glance] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:16:08.458761 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa60f3-673b-4a21-a750-d49fd275ac12 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:16:08.474053 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaa60f3-673b-4a21-a750-d49fd275ac12 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:16:08.475515 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaa60f3-673b-4a21-a750-d49fd275ac12 service glance] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:16:08.489169 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1124/2261] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:16:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:16:08.583395 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe21573a-11de-4263-a871-b422432651d5 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:16:08.583728 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe21573a-11de-4263-a871-b422432651d5 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:16:08.584039 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe21573a-11de-4263-a871-b422432651d5 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:16:08.585850 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1138/2262] 10.222.0.42 () {54 vars in 921 bytes} [Thu Sep 8 01:16:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:16:08.592635 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dfb84d-7311-4229-a051-391fea7deef8 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:16:08.593562 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dfb84d-7311-4229-a051-391fea7deef8 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:16:08.593910 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dfb84d-7311-4229-a051-391fea7deef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:16:08.621861 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-c7dfb84d-7311-4229-a051-391fea7deef8 None None] MFA Rules not processed for user `5dbbcc3dbcc745458e97b31c6781ce70`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:16:08.625336 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-c7dfb84d-7311-4229-a051-391fea7deef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:16:08.642763 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1125/2263] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Sep 8 01:16:08 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:16:08.649572 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe21573a-11de-4263-a871-b422432651d5 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:16:08.651780 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-fe21573a-11de-4263-a871-b422432651d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:16:08.665603 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe21573a-11de-4263-a871-b422432651d5 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:16:08.669573 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a319c4-71dd-4f12-bc91-40b142b62190 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:16:08.670203 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a319c4-71dd-4f12-bc91-40b142b62190 service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:16:08.670520 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a319c4-71dd-4f12-bc91-40b142b62190 service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:16:08.671007 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a319c4-71dd-4f12-bc91-40b142b62190 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:16:08.680736 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a319c4-71dd-4f12-bc91-40b142b62190 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:16:08.682742 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a319c4-71dd-4f12-bc91-40b142b62190 service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:16:08.695322 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1139/2264] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:16:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:16:08.792376 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f5df7-647c-4293-bf8a-124788e081c1 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:16:08.792735 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f5df7-647c-4293-bf8a-124788e081c1 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:16:08.793107 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f5df7-647c-4293-bf8a-124788e081c1 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:16:08.794813 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1126/2265] 10.222.0.42 () {54 vars in 921 bytes} [Thu Sep 8 01:16:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:16:08.800311 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093cb61-d4ab-4c39-9837-653c49ddec7a None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:16:08.800777 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093cb61-d4ab-4c39-9837-653c49ddec7a None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:16:08.800886 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093cb61-d4ab-4c39-9837-653c49ddec7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:16:08.823250 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-a093cb61-d4ab-4c39-9837-653c49ddec7a None None] MFA Rules not processed for user `ae643cfdcb164aa9b474747cd21aa116`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:16:08.824576 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-a093cb61-d4ab-4c39-9837-653c49ddec7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:16:08.835675 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1140/2266] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Sep 8 01:16:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:16:08.846438 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6f5df7-647c-4293-bf8a-124788e081c1 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:16:08.858120 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6f5df7-647c-4293-bf8a-124788e081c1 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:16:08.862230 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a40c5d-74c8-4a6f-894a-e20adbb50dbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:16:08.863401 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a40c5d-74c8-4a6f-894a-e20adbb50dbe service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:16:08.863884 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a40c5d-74c8-4a6f-894a-e20adbb50dbe service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:16:08.864196 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a40c5d-74c8-4a6f-894a-e20adbb50dbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:16:08.875605 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a40c5d-74c8-4a6f-894a-e20adbb50dbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:16:08.877877 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a40c5d-74c8-4a6f-894a-e20adbb50dbe service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:16:08.890152 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1127/2267] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:16:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:16:08.939627 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cffdb6-49f6-4c99-93c8-884867e5a8c5 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:16:08.939953 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cffdb6-49f6-4c99-93c8-884867e5a8c5 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:16:08.940299 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cffdb6-49f6-4c99-93c8-884867e5a8c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:16:09.002890 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-58cffdb6-49f6-4c99-93c8-884867e5a8c5 None None] MFA Rules not processed for user `336e7f07cd7d461c8257995a772ac1f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:16:09.004820 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-58cffdb6-49f6-4c99-93c8-884867e5a8c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:16:09.026414 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:16:09.117654 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1141/2268] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:16:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:16:09.130712 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a40c5d-74c8-4a6f-894a-e20adbb50dbe service neutron] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:16:09.132325 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-38a40c5d-74c8-4a6f-894a-e20adbb50dbe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:16:09.215224 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a40c5d-74c8-4a6f-894a-e20adbb50dbe service neutron] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:16:09.221625 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb0c10a-7b67-4fd2-8b06-9fb8bf87d24c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:16:09.221625 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0c10a-7b67-4fd2-8b06-9fb8bf87d24c service cinder] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:16:09.221625 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0c10a-7b67-4fd2-8b06-9fb8bf87d24c service cinder] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:16:09.221956 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0c10a-7b67-4fd2-8b06-9fb8bf87d24c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:16:09.234307 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb0c10a-7b67-4fd2-8b06-9fb8bf87d24c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:16:09.236585 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb0c10a-7b67-4fd2-8b06-9fb8bf87d24c service cinder] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:16:09.249489 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1128/2269] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:16:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:16:10.109362 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cffdb6-49f6-4c99-93c8-884867e5a8c5 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:16:10.119116 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cffdb6-49f6-4c99-93c8-884867e5a8c5 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:16:10.123307 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a6367-1402-40db-b5e8-dd102e6ca8e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:16:10.124166 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a6367-1402-40db-b5e8-dd102e6ca8e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:16:10.124450 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a6367-1402-40db-b5e8-dd102e6ca8e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:16:10.124727 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a6367-1402-40db-b5e8-dd102e6ca8e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:16:10.126495 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-0a6a6367-1402-40db-b5e8-dd102e6ca8e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:16:10.191289 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6a6367-1402-40db-b5e8-dd102e6ca8e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:16:10.192686 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6a6367-1402-40db-b5e8-dd102e6ca8e2 service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:16:10.202852 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1142/2270] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:16:10 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:16:10.692061 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e839b50-2e41-4b15-8030-ed827f234d2e None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:16:10.692740 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e839b50-2e41-4b15-8030-ed827f234d2e None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:16:10.692740 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e839b50-2e41-4b15-8030-ed827f234d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:16:10.718747 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-3e839b50-2e41-4b15-8030-ed827f234d2e None None] MFA Rules not processed for user `336e7f07cd7d461c8257995a772ac1f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:16:10.720611 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-3e839b50-2e41-4b15-8030-ed827f234d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:16:10.740976 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1129/2271] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:16:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:16:15.705808 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec0d3c-75b1-4470-af39-cce1f1d64584 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:16:15.706664 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec0d3c-75b1-4470-af39-cce1f1d64584 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:16:15.707266 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec0d3c-75b1-4470-af39-cce1f1d64584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:16:15.736400 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-b7ec0d3c-75b1-4470-af39-cce1f1d64584 None None] MFA Rules not processed for user `de75fd7d7721433684cafa2e6d3faaae`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:16:15.738926 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-b7ec0d3c-75b1-4470-af39-cce1f1d64584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:16:15.830952 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1143/2272] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Sep 8 01:16:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:16:15.860357 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e839b50-2e41-4b15-8030-ed827f234d2e None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:16:15.862272 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-3e839b50-2e41-4b15-8030-ed827f234d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:16:15.967597 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e839b50-2e41-4b15-8030-ed827f234d2e None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:16:15.987589 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88342e59-efbf-437f-bc6b-065ff1049b7c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8add26872fe6479aa9d7de155a17195a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:16:15.990053 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88342e59-efbf-437f-bc6b-065ff1049b7c service swift] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:16:15.990516 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88342e59-efbf-437f-bc6b-065ff1049b7c service swift] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:16:16.004706 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88342e59-efbf-437f-bc6b-065ff1049b7c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:16:16.028047 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88342e59-efbf-437f-bc6b-065ff1049b7c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:16:16.030047 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88342e59-efbf-437f-bc6b-065ff1049b7c service swift] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:16:16.046899 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1130/2273] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Sep 8 01:16:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 01:16:25.823280 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=74 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:16:26.036389 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=95 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:16:32.575193 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ec0d3c-75b1-4470-af39-cce1f1d64584 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:16:32.577441 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-b7ec0d3c-75b1-4470-af39-cce1f1d64584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:16:32.656682 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ec0d3c-75b1-4470-af39-cce1f1d64584 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:16:32.662214 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd51fbe-b4ac-47ed-a035-172d7964b474 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '301d30bbd3de4cc69e96bb12729442c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:16:32.663025 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd51fbe-b4ac-47ed-a035-172d7964b474 service placement] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:16:32.663391 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd51fbe-b4ac-47ed-a035-172d7964b474 service placement] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:16:32.664238 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd51fbe-b4ac-47ed-a035-172d7964b474 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:16:32.666546 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-4dd51fbe-b4ac-47ed-a035-172d7964b474 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:16:32.679659 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd51fbe-b4ac-47ed-a035-172d7964b474 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:16:32.681321 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd51fbe-b4ac-47ed-a035-172d7964b474 service placement] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:16:32.693957 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1144/2274] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:16:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:16:40.002041 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a9ed60-e381-4826-99f9-c97e1d70edc0 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:16:40.002595 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a9ed60-e381-4826-99f9-c97e1d70edc0 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:16:40.002595 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a9ed60-e381-4826-99f9-c97e1d70edc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:16:40.023290 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-65a9ed60-e381-4826-99f9-c97e1d70edc0 None None] MFA Rules not processed for user `de75fd7d7721433684cafa2e6d3faaae`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:16:40.024630 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-65a9ed60-e381-4826-99f9-c97e1d70edc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:16:40.032919 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1131/2275] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Sep 8 01:16:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:16:40.047744 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd51fbe-b4ac-47ed-a035-172d7964b474 service placement] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:16:40.059720 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd51fbe-b4ac-47ed-a035-172d7964b474 service placement] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:16:40.064858 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be61ef-da39-4909-af4c-921b18155b63 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8add26872fe6479aa9d7de155a17195a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:16:40.065452 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be61ef-da39-4909-af4c-921b18155b63 service swift] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:16:40.065732 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be61ef-da39-4909-af4c-921b18155b63 service swift] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:16:40.066010 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be61ef-da39-4909-af4c-921b18155b63 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:16:40.074356 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06be61ef-da39-4909-af4c-921b18155b63 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:16:40.075945 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06be61ef-da39-4909-af4c-921b18155b63 service swift] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:16:40.084101 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1145/2276] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Sep 8 01:16:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 01:16:43.275620 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc3898-bcdd-415f-ab21-a5ca2414fca8 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:16:43.276189 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc3898-bcdd-415f-ab21-a5ca2414fca8 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:16:43.276293 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc3898-bcdd-415f-ab21-a5ca2414fca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:16:43.297788 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-1fbc3898-bcdd-415f-ab21-a5ca2414fca8 None None] MFA Rules not processed for user `83fea9762aaf4766a23f1774ff1411fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:16:43.299187 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-1fbc3898-bcdd-415f-ab21-a5ca2414fca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:16:43.385989 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1132/2277] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:16:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:16:50.080896 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=38 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:16:53.373905 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=28 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:17:03.511233 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e0257-8a9c-4a72-b03c-6bdc669a33d5 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:17:03.511992 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e0257-8a9c-4a72-b03c-6bdc669a33d5 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:17:03.511992 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e0257-8a9c-4a72-b03c-6bdc669a33d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:17:03.533968 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-c47e0257-8a9c-4a72-b03c-6bdc669a33d5 None None] MFA Rules not processed for user `de75fd7d7721433684cafa2e6d3faaae`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:17:03.535180 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-c47e0257-8a9c-4a72-b03c-6bdc669a33d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:17:03.543389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1146/2278] 10.222.0.42 () {58 vars in 967 bytes} [Thu Sep 8 01:17:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:17:03.556783 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbc3898-bcdd-415f-ab21-a5ca2414fca8 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:17:03.570311 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbc3898-bcdd-415f-ab21-a5ca2414fca8 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:17:03.576024 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2099f66-e69c-4bcf-b906-9600b1bea653 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8add26872fe6479aa9d7de155a17195a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:17:03.576883 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2099f66-e69c-4bcf-b906-9600b1bea653 service swift] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:17:03.577176 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2099f66-e69c-4bcf-b906-9600b1bea653 service swift] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:17:03.577445 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2099f66-e69c-4bcf-b906-9600b1bea653 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:17:03.588423 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2099f66-e69c-4bcf-b906-9600b1bea653 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:17:03.590555 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2099f66-e69c-4bcf-b906-9600b1bea653 service swift] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:17:03.600041 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1133/2279] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Sep 8 01:17:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 01:17:07.607420 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0babd2cd-9c41-4187-a427-ef8cb2b4a882 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:17:07.607915 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0babd2cd-9c41-4187-a427-ef8cb2b4a882 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:17:07.607915 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0babd2cd-9c41-4187-a427-ef8cb2b4a882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:17:07.627015 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-0babd2cd-9c41-4187-a427-ef8cb2b4a882 None None] MFA Rules not processed for user `de75fd7d7721433684cafa2e6d3faaae`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:17:07.628189 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-0babd2cd-9c41-4187-a427-ef8cb2b4a882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:17:07.635642 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1147/2280] 10.222.0.42 () {58 vars in 967 bytes} [Thu Sep 8 01:17:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:17:07.648889 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2099f66-e69c-4bcf-b906-9600b1bea653 service swift] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:17:07.660259 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2099f66-e69c-4bcf-b906-9600b1bea653 service swift] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:17:07.665344 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4f394b-06db-4240-b423-89e6cecc31ed service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8add26872fe6479aa9d7de155a17195a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:17:07.665937 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4f394b-06db-4240-b423-89e6cecc31ed service swift] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:17:07.666219 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4f394b-06db-4240-b423-89e6cecc31ed service swift] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:17:07.666491 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4f394b-06db-4240-b423-89e6cecc31ed service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:17:07.674904 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4f394b-06db-4240-b423-89e6cecc31ed service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:17:07.676543 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4f394b-06db-4240-b423-89e6cecc31ed service swift] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:17:07.684599 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1134/2281] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Sep 8 01:17:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 01:17:09.732773 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0babd2cd-9c41-4187-a427-ef8cb2b4a882 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:17:09.734317 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-0babd2cd-9c41-4187-a427-ef8cb2b4a882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:17:09.759995 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0babd2cd-9c41-4187-a427-ef8cb2b4a882 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:17:09.765376 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630a5007-bd14-46f3-9ac4-ec10366db808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:17:09.766232 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a5007-bd14-46f3-9ac4-ec10366db808 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:17:09.766515 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a5007-bd14-46f3-9ac4-ec10366db808 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:17:09.766791 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a5007-bd14-46f3-9ac4-ec10366db808 admin admin] PATH_INFO: `/v3/users/e5351a59a0294f59b6b26ddc22ba7ffa` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:17:09.767934 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630a5007-bd14-46f3-9ac4-ec10366db808 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5351a59a0294f59b6b26ddc22ba7ffa)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:17:09.769909 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630a5007-bd14-46f3-9ac4-ec10366db808 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:17:09.819730 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-630a5007-bd14-46f3-9ac4-ec10366db808 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5351a59a0294f59b6b26ddc22ba7ffa'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:17:09.835540 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-630a5007-bd14-46f3-9ac4-ec10366db808 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5351a59a0294f59b6b26ddc22ba7ffa'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:17:09.843244 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-630a5007-bd14-46f3-9ac4-ec10366db808 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5351a59a0294f59b6b26ddc22ba7ffa'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:17:09.851015 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-630a5007-bd14-46f3-9ac4-ec10366db808 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5351a59a0294f59b6b26ddc22ba7ffa'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:17:09.851643 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-630a5007-bd14-46f3-9ac4-ec10366db808 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5351a59a0294f59b6b26ddc22ba7ffa'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:17:09.859017 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1148/2282] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:17:09 2022] DELETE /identity/v3/users/e5351a59a0294f59b6b26ddc22ba7ffa => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:17:10.033733 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4f394b-06db-4240-b423-89e6cecc31ed service swift] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:17:10.045431 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4f394b-06db-4240-b423-89e6cecc31ed service swift] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:17:10.127132 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1af69a-3fbc-4f15-8c34-7dca0eff7657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:17:10.127795 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1af69a-3fbc-4f15-8c34-7dca0eff7657 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:17:10.128100 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1af69a-3fbc-4f15-8c34-7dca0eff7657 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:17:10.128410 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1af69a-3fbc-4f15-8c34-7dca0eff7657 admin admin] PATH_INFO: `/v3/projects/8263d46be0db47ce9191c099759cf0e9` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:17:10.129663 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1af69a-3fbc-4f15-8c34-7dca0eff7657 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8263d46be0db47ce9191c099759cf0e9)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:17:10.131616 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1af69a-3fbc-4f15-8c34-7dca0eff7657 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:17:10.183031 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-1b1af69a-3fbc-4f15-8c34-7dca0eff7657 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8263d46be0db47ce9191c099759cf0e9', 'request_id': 'req-1b1af69a-3fbc-4f15-8c34-7dca0eff7657'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:17:10.189952 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-1b1af69a-3fbc-4f15-8c34-7dca0eff7657 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8263d46be0db47ce9191c099759cf0e9', 'request_id': 'req-1b1af69a-3fbc-4f15-8c34-7dca0eff7657'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:17:10.196307 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-1b1af69a-3fbc-4f15-8c34-7dca0eff7657 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8263d46be0db47ce9191c099759cf0e9', 'request_id': 'req-1b1af69a-3fbc-4f15-8c34-7dca0eff7657'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:17:10.207078 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-1b1af69a-3fbc-4f15-8c34-7dca0eff7657 admin admin] The token cache is being invalidate because project 8263d46be0db47ce9191c099759cf0e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81661) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:17:10.207713 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-1b1af69a-3fbc-4f15-8c34-7dca0eff7657 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:17:10.208036 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-1b1af69a-3fbc-4f15-8c34-7dca0eff7657 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:17:10.209665 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1135/2283] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:17:10 2022] DELETE /identity/v3/projects/8263d46be0db47ce9191c099759cf0e9 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:17:10.216495 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab7bb7-9d2a-4f2d-95a8-5d25d4d94eff None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:17:10.216887 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab7bb7-9d2a-4f2d-95a8-5d25d4d94eff None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:17:10.217377 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab7bb7-9d2a-4f2d-95a8-5d25d4d94eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:17:10.239277 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-4eab7bb7-9d2a-4f2d-95a8-5d25d4d94eff None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:17:10.240544 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-4eab7bb7-9d2a-4f2d-95a8-5d25d4d94eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:17:10.315057 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1149/2284] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:17:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:17:10.316890 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93fd82-417b-46f1-ae11-9d7192945966 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:17:10.317599 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93fd82-417b-46f1-ae11-9d7192945966 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:17:10.317599 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93fd82-417b-46f1-ae11-9d7192945966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:17:10.338513 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-ed93fd82-417b-46f1-ae11-9d7192945966 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:17:10.339820 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-ed93fd82-417b-46f1-ae11-9d7192945966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:17:10.348351 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1136/2285] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:17:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:17:10.352551 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eab7bb7-9d2a-4f2d-95a8-5d25d4d94eff None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:17:10.362350 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eab7bb7-9d2a-4f2d-95a8-5d25d4d94eff None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:17:10.367023 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba7bd4-0756-4204-9c12-fca3ddd6c0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:17:10.367613 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba7bd4-0756-4204-9c12-fca3ddd6c0b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:17:10.367893 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba7bd4-0756-4204-9c12-fca3ddd6c0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:17:10.368196 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba7bd4-0756-4204-9c12-fca3ddd6c0b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:17:10.368843 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ba7bd4-0756-4204-9c12-fca3ddd6c0b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:17:10.369152 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ba7bd4-0756-4204-9c12-fca3ddd6c0b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:17:10.371260 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ba7bd4-0756-4204-9c12-fca3ddd6c0b2 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:17:10.382968 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1150/2286] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:17:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:17:10.388617 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed93fd82-417b-46f1-ae11-9d7192945966 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:17:10.398793 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed93fd82-417b-46f1-ae11-9d7192945966 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:17:10.403947 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f66ea1-7e3c-41b7-b091-86df10d9664d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:17:10.404831 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f66ea1-7e3c-41b7-b091-86df10d9664d admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:17:10.405111 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f66ea1-7e3c-41b7-b091-86df10d9664d admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:17:10.405377 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f66ea1-7e3c-41b7-b091-86df10d9664d admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:17:10.406002 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f66ea1-7e3c-41b7-b091-86df10d9664d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:17:10.408064 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f66ea1-7e3c-41b7-b091-86df10d9664d admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:17:10.442426 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1137/2287] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:17:10 2022] POST /identity/v3/projects => generated 399 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 01:17:10.443821 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba7bd4-0756-4204-9c12-fca3ddd6c0b2 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:17:10.455371 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba7bd4-0756-4204-9c12-fca3ddd6c0b2 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:17:10.506179 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:17:10.526268 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f85e5e0-0ed0-47c9-b3a9-80a4f36d6e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:17:10.526268 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85e5e0-0ed0-47c9-b3a9-80a4f36d6e4d admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:17:10.526268 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85e5e0-0ed0-47c9-b3a9-80a4f36d6e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:17:10.526268 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85e5e0-0ed0-47c9-b3a9-80a4f36d6e4d admin admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:17:10.526646 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f85e5e0-0ed0-47c9-b3a9-80a4f36d6e4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:17:10.526646 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f85e5e0-0ed0-47c9-b3a9-80a4f36d6e4d admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:17:10.556227 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1151/2288] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:17:10 2022] POST /identity/v3/users => generated 300 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:17:10.560855 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f66ea1-7e3c-41b7-b091-86df10d9664d admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:17:10.572743 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f66ea1-7e3c-41b7-b091-86df10d9664d admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:17:10.578174 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bcf00b-a895-4a21-a377-7c97da891c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:17:10.578338 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bcf00b-a895-4a21-a377-7c97da891c22 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:17:10.578523 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bcf00b-a895-4a21-a377-7c97da891c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:17:10.578788 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bcf00b-a895-4a21-a377-7c97da891c22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:17:10.579430 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bcf00b-a895-4a21-a377-7c97da891c22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:17:10.581069 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bcf00b-a895-4a21-a377-7c97da891c22 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:17:10.606114 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1138/2289] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:17:10 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:17:10.609877 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f85e5e0-0ed0-47c9-b3a9-80a4f36d6e4d admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:17:10.625158 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f85e5e0-0ed0-47c9-b3a9-80a4f36d6e4d admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:17:10.632931 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2570865b-6d26-407b-a17b-0bbbd3b42578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:17:10.632931 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570865b-6d26-407b-a17b-0bbbd3b42578 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:17:10.632931 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570865b-6d26-407b-a17b-0bbbd3b42578 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:17:10.633307 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570865b-6d26-407b-a17b-0bbbd3b42578 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:17:10.634246 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2570865b-6d26-407b-a17b-0bbbd3b42578 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:17:10.636741 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2570865b-6d26-407b-a17b-0bbbd3b42578 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:17:10.650531 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1152/2290] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:17:10 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:17:10.654466 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bcf00b-a895-4a21-a377-7c97da891c22 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:17:10.666352 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bcf00b-a895-4a21-a377-7c97da891c22 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:17:10.672426 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f0ba84-3174-428c-a8aa-c5ee4c191900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:17:10.673199 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f0ba84-3174-428c-a8aa-c5ee4c191900 admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:17:10.673349 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f0ba84-3174-428c-a8aa-c5ee4c191900 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:17:10.673616 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f0ba84-3174-428c-a8aa-c5ee4c191900 admin admin] PATH_INFO: `/v3/projects/a73ab367a4bb42259d302bdf5e64739b/users/c0b0cf89034f4ae88868f5b3dd1c77ad/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:17:10.689670 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f0ba84-3174-428c-a8aa-c5ee4c191900 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a73ab367a4bb42259d302bdf5e64739b, user_id=c0b0cf89034f4ae88868f5b3dd1c77ad, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:17:10.692426 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f0ba84-3174-428c-a8aa-c5ee4c191900 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:17:10.701434 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:17:10.707020 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1139/2291] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:17:10 2022] PUT /identity/v3/projects/a73ab367a4bb42259d302bdf5e64739b/users/c0b0cf89034f4ae88868f5b3dd1c77ad/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:17:10.720781 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb1bbd-beda-429d-ba4b-d3a74c083720 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:17:10.721070 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb1bbd-beda-429d-ba4b-d3a74c083720 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:17:10.721341 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb1bbd-beda-429d-ba4b-d3a74c083720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:17:10.750101 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-c5bb1bbd-beda-429d-ba4b-d3a74c083720 None None] MFA Rules not processed for user `c0b0cf89034f4ae88868f5b3dd1c77ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:17:10.751310 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-c5bb1bbd-beda-429d-ba4b-d3a74c083720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:17:10.815796 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1153/2292] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:17:10 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:17:10.829471 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f0ba84-3174-428c-a8aa-c5ee4c191900 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:17:10.831205 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-b5f0ba84-3174-428c-a8aa-c5ee4c191900 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:17:10.887648 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f0ba84-3174-428c-a8aa-c5ee4c191900 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:17:10.891340 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b24f64-5a7b-4b48-8bbb-2069ce7fe8e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:17:10.891932 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b24f64-5a7b-4b48-8bbb-2069ce7fe8e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:17:10.892241 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b24f64-5a7b-4b48-8bbb-2069ce7fe8e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:17:10.892511 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b24f64-5a7b-4b48-8bbb-2069ce7fe8e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:17:10.900842 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b24f64-5a7b-4b48-8bbb-2069ce7fe8e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:17:10.902882 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b24f64-5a7b-4b48-8bbb-2069ce7fe8e2 service cinder] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:17:10.911738 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1140/2293] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:17:10 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:17:19.092589 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb1bbd-beda-429d-ba4b-d3a74c083720 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:17:19.161980 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb1bbd-beda-429d-ba4b-d3a74c083720 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:17:19.166568 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291771fb-76b0-4683-bd5c-67e1ab90be4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:17:19.167231 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291771fb-76b0-4683-bd5c-67e1ab90be4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:17:19.167515 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291771fb-76b0-4683-bd5c-67e1ab90be4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:17:19.167796 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291771fb-76b0-4683-bd5c-67e1ab90be4b admin admin] PATH_INFO: `/v3/users/c0b0cf89034f4ae88868f5b3dd1c77ad` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:17:19.168987 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291771fb-76b0-4683-bd5c-67e1ab90be4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0b0cf89034f4ae88868f5b3dd1c77ad)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:17:19.170932 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291771fb-76b0-4683-bd5c-67e1ab90be4b admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:17:19.222125 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-291771fb-76b0-4683-bd5c-67e1ab90be4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0b0cf89034f4ae88868f5b3dd1c77ad'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:17:19.230804 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:17:19.238918 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-291771fb-76b0-4683-bd5c-67e1ab90be4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0b0cf89034f4ae88868f5b3dd1c77ad'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:17:19.247244 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-291771fb-76b0-4683-bd5c-67e1ab90be4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0b0cf89034f4ae88868f5b3dd1c77ad'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:17:19.255186 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-291771fb-76b0-4683-bd5c-67e1ab90be4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0b0cf89034f4ae88868f5b3dd1c77ad'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:17:19.255885 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-291771fb-76b0-4683-bd5c-67e1ab90be4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0b0cf89034f4ae88868f5b3dd1c77ad'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:17:19.268857 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1154/2294] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:17:19 2022] DELETE /identity/v3/users/c0b0cf89034f4ae88868f5b3dd1c77ad => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:17:19.273324 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16396400-6756-4df9-8de9-d44ed8850686 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:17:19.273947 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16396400-6756-4df9-8de9-d44ed8850686 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:17:19.273947 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16396400-6756-4df9-8de9-d44ed8850686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:17:19.299359 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-16396400-6756-4df9-8de9-d44ed8850686 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:17:19.300807 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-16396400-6756-4df9-8de9-d44ed8850686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:17:19.382228 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1141/2295] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:17:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:17:19.409625 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291771fb-76b0-4683-bd5c-67e1ab90be4b admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:17:19.411993 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-291771fb-76b0-4683-bd5c-67e1ab90be4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:17:19.484592 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291771fb-76b0-4683-bd5c-67e1ab90be4b admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:17:19.488522 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a79f8fd-2044-43e2-9506-152c55416736 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:17:19.489236 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a79f8fd-2044-43e2-9506-152c55416736 service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:17:19.489519 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a79f8fd-2044-43e2-9506-152c55416736 service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:17:19.489796 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a79f8fd-2044-43e2-9506-152c55416736 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:17:19.499799 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a79f8fd-2044-43e2-9506-152c55416736 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:17:19.501299 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a79f8fd-2044-43e2-9506-152c55416736 service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:17:19.511270 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1155/2296] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:17:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 01:17:19.801008 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16396400-6756-4df9-8de9-d44ed8850686 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:17:19.802463 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-16396400-6756-4df9-8de9-d44ed8850686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:17:19.819754 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16396400-6756-4df9-8de9-d44ed8850686 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:17:19.825602 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cc7216-6f95-4749-89ed-d0acf5672dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:17:19.826253 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc7216-6f95-4749-89ed-d0acf5672dcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:17:19.826523 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc7216-6f95-4749-89ed-d0acf5672dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:17:19.826794 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc7216-6f95-4749-89ed-d0acf5672dcc admin admin] PATH_INFO: `/v3/projects/a73ab367a4bb42259d302bdf5e64739b` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:17:19.828090 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cc7216-6f95-4749-89ed-d0acf5672dcc admin admin] RBAC: Authorizing `identity:delete_project(project_id=a73ab367a4bb42259d302bdf5e64739b)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:17:19.829916 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cc7216-6f95-4749-89ed-d0acf5672dcc admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:17:19.881300 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-87cc7216-6f95-4749-89ed-d0acf5672dcc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a73ab367a4bb42259d302bdf5e64739b', 'request_id': 'req-87cc7216-6f95-4749-89ed-d0acf5672dcc'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:17:19.890028 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-87cc7216-6f95-4749-89ed-d0acf5672dcc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a73ab367a4bb42259d302bdf5e64739b', 'request_id': 'req-87cc7216-6f95-4749-89ed-d0acf5672dcc'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:17:19.896708 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-87cc7216-6f95-4749-89ed-d0acf5672dcc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a73ab367a4bb42259d302bdf5e64739b', 'request_id': 'req-87cc7216-6f95-4749-89ed-d0acf5672dcc'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:17:19.907289 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-87cc7216-6f95-4749-89ed-d0acf5672dcc admin admin] The token cache is being invalidate because project a73ab367a4bb42259d302bdf5e64739b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81661) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:17:19.907578 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-87cc7216-6f95-4749-89ed-d0acf5672dcc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:17:19.908344 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-87cc7216-6f95-4749-89ed-d0acf5672dcc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:17:19.910052 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1142/2297] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:17:19 2022] DELETE /identity/v3/projects/a73ab367a4bb42259d302bdf5e64739b => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:17:19.915117 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd6e49-2b01-42ac-ae49-bdc6fce78954 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:17:19.915537 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd6e49-2b01-42ac-ae49-bdc6fce78954 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:17:19.915893 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd6e49-2b01-42ac-ae49-bdc6fce78954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:17:19.936992 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-21bd6e49-2b01-42ac-ae49-bdc6fce78954 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:17:19.938335 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-21bd6e49-2b01-42ac-ae49-bdc6fce78954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:17:20.004363 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1156/2298] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:17:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:17:20.012833 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff7733-8ce7-4e8b-982b-a2fdc896738c None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:17:20.013142 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff7733-8ce7-4e8b-982b-a2fdc896738c None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:17:20.013439 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff7733-8ce7-4e8b-982b-a2fdc896738c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:17:20.033795 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-b3ff7733-8ce7-4e8b-982b-a2fdc896738c None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:17:20.035009 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-b3ff7733-8ce7-4e8b-982b-a2fdc896738c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:17:20.043286 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1143/2299] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:17:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:17:20.047647 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd6e49-2b01-42ac-ae49-bdc6fce78954 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:17:20.057396 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd6e49-2b01-42ac-ae49-bdc6fce78954 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:17:20.063415 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d511b9a9-397f-480f-9d10-52035fa96065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:17:20.064125 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511b9a9-397f-480f-9d10-52035fa96065 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:17:20.064351 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511b9a9-397f-480f-9d10-52035fa96065 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:17:20.064545 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511b9a9-397f-480f-9d10-52035fa96065 admin admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:17:20.065166 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d511b9a9-397f-480f-9d10-52035fa96065 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:17:20.065471 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d511b9a9-397f-480f-9d10-52035fa96065 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:17:20.067106 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d511b9a9-397f-480f-9d10-52035fa96065 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:17:20.082590 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1157/2300] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:17:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:17:20.084566 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff7733-8ce7-4e8b-982b-a2fdc896738c None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:17:20.094078 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff7733-8ce7-4e8b-982b-a2fdc896738c None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:17:20.098842 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4edb76-5258-47fb-a32c-45a64648bf18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:17:20.099438 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4edb76-5258-47fb-a32c-45a64648bf18 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:17:20.099719 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4edb76-5258-47fb-a32c-45a64648bf18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:17:20.099992 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4edb76-5258-47fb-a32c-45a64648bf18 admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:17:20.100667 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4edb76-5258-47fb-a32c-45a64648bf18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:17:20.102583 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4edb76-5258-47fb-a32c-45a64648bf18 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:17:20.123422 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:17:20.131565 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1144/2301] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:17:20 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 01:17:20.136567 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d511b9a9-397f-480f-9d10-52035fa96065 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:17:20.146470 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d511b9a9-397f-480f-9d10-52035fa96065 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:17:20.206468 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79faec2f-756c-459b-b6b1-487a98daba1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:17:20.207073 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79faec2f-756c-459b-b6b1-487a98daba1d admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:17:20.207353 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79faec2f-756c-459b-b6b1-487a98daba1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:17:20.207626 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79faec2f-756c-459b-b6b1-487a98daba1d admin admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:17:20.208322 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79faec2f-756c-459b-b6b1-487a98daba1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:17:20.210243 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79faec2f-756c-459b-b6b1-487a98daba1d admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:17:20.220159 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:17:20.233142 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1158/2302] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:17:20 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:17:20.237266 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4edb76-5258-47fb-a32c-45a64648bf18 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:17:20.248826 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4edb76-5258-47fb-a32c-45a64648bf18 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:17:20.253783 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33891957-e621-4dab-b6a3-4aea4958abb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:17:20.254402 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33891957-e621-4dab-b6a3-4aea4958abb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:17:20.254677 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33891957-e621-4dab-b6a3-4aea4958abb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:17:20.254945 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33891957-e621-4dab-b6a3-4aea4958abb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:17:20.255577 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33891957-e621-4dab-b6a3-4aea4958abb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:17:20.257202 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33891957-e621-4dab-b6a3-4aea4958abb0 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:17:20.268472 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1145/2303] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:17:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:17:20.272466 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79faec2f-756c-459b-b6b1-487a98daba1d admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:17:20.282386 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79faec2f-756c-459b-b6b1-487a98daba1d admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:17:20.287019 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d50e68-637e-4bed-be54-e65a6458aa0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:17:20.287609 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d50e68-637e-4bed-be54-e65a6458aa0c admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:17:20.287884 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d50e68-637e-4bed-be54-e65a6458aa0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:17:20.288203 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d50e68-637e-4bed-be54-e65a6458aa0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:17:20.288840 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d50e68-637e-4bed-be54-e65a6458aa0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:17:20.290417 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d50e68-637e-4bed-be54-e65a6458aa0c admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:17:20.301805 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1159/2304] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:17:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:17:20.305998 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33891957-e621-4dab-b6a3-4aea4958abb0 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:17:20.317290 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33891957-e621-4dab-b6a3-4aea4958abb0 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:17:20.321910 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeff6dcf-6ba1-4d12-aa2d-bd1755c36d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:17:20.322560 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff6dcf-6ba1-4d12-aa2d-bd1755c36d29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:17:20.322839 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff6dcf-6ba1-4d12-aa2d-bd1755c36d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:17:20.323242 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff6dcf-6ba1-4d12-aa2d-bd1755c36d29 admin admin] PATH_INFO: `/v3/projects/4221b10efb3d4bb799d820e985a48651/users/744fe07ea5644157bdbf11d81adbab89/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:17:20.338202 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeff6dcf-6ba1-4d12-aa2d-bd1755c36d29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4221b10efb3d4bb799d820e985a48651, user_id=744fe07ea5644157bdbf11d81adbab89, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:17:20.340892 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeff6dcf-6ba1-4d12-aa2d-bd1755c36d29 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:17:20.353621 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1146/2305] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:17:20 2022] PUT /identity/v3/projects/4221b10efb3d4bb799d820e985a48651/users/744fe07ea5644157bdbf11d81adbab89/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:17:20.359102 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b57024-16de-4167-afab-e09955189dbb None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:17:20.359385 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b57024-16de-4167-afab-e09955189dbb None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:17:20.359656 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b57024-16de-4167-afab-e09955189dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:17:20.379560 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-98b57024-16de-4167-afab-e09955189dbb None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:17:20.380794 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-98b57024-16de-4167-afab-e09955189dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:17:20.439926 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1160/2306] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:17:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:17:20.449691 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeff6dcf-6ba1-4d12-aa2d-bd1755c36d29 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:17:20.451371 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-eeff6dcf-6ba1-4d12-aa2d-bd1755c36d29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:17:20.507726 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeff6dcf-6ba1-4d12-aa2d-bd1755c36d29 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:17:20.512090 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84823a09-a277-43cb-beac-6e54a7a3dec2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:17:20.512723 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84823a09-a277-43cb-beac-6e54a7a3dec2 service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:17:20.513021 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84823a09-a277-43cb-beac-6e54a7a3dec2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:17:20.513292 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84823a09-a277-43cb-beac-6e54a7a3dec2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:17:20.522541 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84823a09-a277-43cb-beac-6e54a7a3dec2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:17:20.523960 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84823a09-a277-43cb-beac-6e54a7a3dec2 service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:17:20.533666 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1147/2307] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:17:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:17:24.848508 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cad549-b183-466f-9919-94ce61709796 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:17:24.848984 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cad549-b183-466f-9919-94ce61709796 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:17:24.849093 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cad549-b183-466f-9919-94ce61709796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:17:24.878338 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-f4cad549-b183-466f-9919-94ce61709796 None None] MFA Rules not processed for user `744fe07ea5644157bdbf11d81adbab89`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:17:24.879467 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-f4cad549-b183-466f-9919-94ce61709796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:17:24.951764 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1161/2308] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:17:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:17:24.956311 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84823a09-a277-43cb-beac-6e54a7a3dec2 service neutron] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:17:24.966649 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84823a09-a277-43cb-beac-6e54a7a3dec2 service neutron] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:17:24.971635 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90674066-2b49-43f3-b0ca-3c831d468ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:17:24.972292 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90674066-2b49-43f3-b0ca-3c831d468ce3 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:17:24.972576 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90674066-2b49-43f3-b0ca-3c831d468ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:17:24.972853 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90674066-2b49-43f3-b0ca-3c831d468ce3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:17:24.973507 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90674066-2b49-43f3-b0ca-3c831d468ce3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:17:24.975466 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90674066-2b49-43f3-b0ca-3c831d468ce3 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:17:25.007652 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1148/2309] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:17:24 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:17:25.009116 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cad549-b183-466f-9919-94ce61709796 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:17:25.020663 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cad549-b183-466f-9919-94ce61709796 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:17:25.080678 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b63523b-3977-4bc2-9e44-4ff8f219a038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:17:25.081445 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b63523b-3977-4bc2-9e44-4ff8f219a038 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:17:25.081863 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b63523b-3977-4bc2-9e44-4ff8f219a038 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:17:25.082255 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b63523b-3977-4bc2-9e44-4ff8f219a038 admin admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:17:25.082994 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b63523b-3977-4bc2-9e44-4ff8f219a038 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:17:25.084992 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b63523b-3977-4bc2-9e44-4ff8f219a038 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:17:25.112524 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1162/2310] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:17:25 2022] POST /identity/v3/users => generated 295 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 01:17:25.114137 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90674066-2b49-43f3-b0ca-3c831d468ce3 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:17:25.125458 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90674066-2b49-43f3-b0ca-3c831d468ce3 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:17:25.130084 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b30a91-eba6-4c9b-be5d-636aaa6660f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:17:25.130689 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b30a91-eba6-4c9b-be5d-636aaa6660f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:17:25.130965 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b30a91-eba6-4c9b-be5d-636aaa6660f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:17:25.131234 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b30a91-eba6-4c9b-be5d-636aaa6660f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:17:25.131876 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b30a91-eba6-4c9b-be5d-636aaa6660f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:17:25.133514 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b30a91-eba6-4c9b-be5d-636aaa6660f6 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:17:25.145347 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1149/2311] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:17:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:17:25.149543 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b63523b-3977-4bc2-9e44-4ff8f219a038 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:17:25.155678 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:17:25.161327 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b63523b-3977-4bc2-9e44-4ff8f219a038 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:17:25.166718 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c7ad92-6bb0-49ac-9c2a-e6b15c239c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:17:25.168388 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c7ad92-6bb0-49ac-9c2a-e6b15c239c21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:17:25.168679 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c7ad92-6bb0-49ac-9c2a-e6b15c239c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:17:25.168976 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c7ad92-6bb0-49ac-9c2a-e6b15c239c21 admin admin] PATH_INFO: `/v3/projects/5905339d520b40308156ab0533803bde/users/14299221fc9d4ef1a202b08cdfd51503/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:17:25.184998 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c7ad92-6bb0-49ac-9c2a-e6b15c239c21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5905339d520b40308156ab0533803bde, user_id=14299221fc9d4ef1a202b08cdfd51503, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:17:25.187693 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c7ad92-6bb0-49ac-9c2a-e6b15c239c21 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:17:25.202682 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1163/2312] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:17:25 2022] PUT /identity/v3/projects/5905339d520b40308156ab0533803bde/users/14299221fc9d4ef1a202b08cdfd51503/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:17:25.206575 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b30a91-eba6-4c9b-be5d-636aaa6660f6 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:17:25.217935 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b30a91-eba6-4c9b-be5d-636aaa6660f6 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:17:25.289347 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15988011-0d24-454b-aa24-95155d341e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:17:25.289972 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15988011-0d24-454b-aa24-95155d341e56 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:17:25.290255 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15988011-0d24-454b-aa24-95155d341e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:17:25.290530 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15988011-0d24-454b-aa24-95155d341e56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:17:25.291180 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15988011-0d24-454b-aa24-95155d341e56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:17:25.293534 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15988011-0d24-454b-aa24-95155d341e56 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:17:25.306517 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1150/2313] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:17:25 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:17:25.310825 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c7ad92-6bb0-49ac-9c2a-e6b15c239c21 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:17:25.330769 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c7ad92-6bb0-49ac-9c2a-e6b15c239c21 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:17:25.338645 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa8719d-76ef-4842-b032-44488d4d15cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:17:25.340339 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa8719d-76ef-4842-b032-44488d4d15cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:17:25.340620 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa8719d-76ef-4842-b032-44488d4d15cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:17:25.340888 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa8719d-76ef-4842-b032-44488d4d15cd admin admin] PATH_INFO: `/v3/projects/5905339d520b40308156ab0533803bde/users/14299221fc9d4ef1a202b08cdfd51503/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:17:25.342893 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa8719d-76ef-4842-b032-44488d4d15cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=5905339d520b40308156ab0533803bde, user_id=14299221fc9d4ef1a202b08cdfd51503, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:17:25.345607 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa8719d-76ef-4842-b032-44488d4d15cd admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:17:25.364785 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1164/2314] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:17:25 2022] PUT /identity/v3/projects/5905339d520b40308156ab0533803bde/users/14299221fc9d4ef1a202b08cdfd51503/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:17:30.456439 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d8939-a4bc-48fb-ba5e-40371a81615d None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:17:30.457980 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d8939-a4bc-48fb-ba5e-40371a81615d None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:17:30.458438 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d8939-a4bc-48fb-ba5e-40371a81615d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:17:30.509808 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-943d8939-a4bc-48fb-ba5e-40371a81615d None None] MFA Rules not processed for user `14299221fc9d4ef1a202b08cdfd51503`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:17:30.511677 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-943d8939-a4bc-48fb-ba5e-40371a81615d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:17:30.538731 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:17:30.607128 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1151/2315] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:17:30 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:17:30.625250 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa8719d-76ef-4842-b032-44488d4d15cd admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:17:30.627740 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-7fa8719d-76ef-4842-b032-44488d4d15cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:17:30.751367 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa8719d-76ef-4842-b032-44488d4d15cd admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:17:30.758260 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949de978-a243-4c4b-ac19-e9c04f104c99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae643cfdcb164aa9b474747cd21aa116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:17:30.759250 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949de978-a243-4c4b-ac19-e9c04f104c99 service nova] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:17:30.759711 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949de978-a243-4c4b-ac19-e9c04f104c99 service nova] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:17:30.759872 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949de978-a243-4c4b-ac19-e9c04f104c99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:17:30.833548 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949de978-a243-4c4b-ac19-e9c04f104c99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:17:30.834547 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949de978-a243-4c4b-ac19-e9c04f104c99 service nova] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:17:30.859139 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1165/2316] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:17:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:17:40.603443 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=21 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:17:40.842775 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=70,keystone:INSERT=2 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:17:45.619005 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa982e-b98c-4b74-9641-1427d4f9ca28 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:17:45.619496 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa982e-b98c-4b74-9641-1427d4f9ca28 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:17:45.619595 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa982e-b98c-4b74-9641-1427d4f9ca28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:17:45.639786 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-c1aa982e-b98c-4b74-9641-1427d4f9ca28 None None] MFA Rules not processed for user `20af4425b7434d2dab7baeacbb845387`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:17:45.641186 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-c1aa982e-b98c-4b74-9641-1427d4f9ca28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:17:45.701502 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1152/2317] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:17:45 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:17:55.137816 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c74923d-8d6a-4329-a3ab-ae9cf7a9c1ac None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:17:55.138260 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c74923d-8d6a-4329-a3ab-ae9cf7a9c1ac None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:17:55.138260 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c74923d-8d6a-4329-a3ab-ae9cf7a9c1ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:17:55.157646 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-1c74923d-8d6a-4329-a3ab-ae9cf7a9c1ac None None] MFA Rules not processed for user `744fe07ea5644157bdbf11d81adbab89`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:17:55.158845 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-1c74923d-8d6a-4329-a3ab-ae9cf7a9c1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:17:55.165702 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1166/2318] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:17:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:17:55.693934 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=25 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:18:04.936853 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e8f83-4976-494f-90fc-ae849b116381 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:18:04.937514 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e8f83-4976-494f-90fc-ae849b116381 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:18:04.937514 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e8f83-4976-494f-90fc-ae849b116381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:18:04.958016 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-321e8f83-4976-494f-90fc-ae849b116381 None None] MFA Rules not processed for user `336e7f07cd7d461c8257995a772ac1f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:18:04.959241 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-321e8f83-4976-494f-90fc-ae849b116381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:18:05.019031 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1153/2319] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:18:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:18:05.151645 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=3 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:18:12.230567 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4808fa-768b-4c67-a51a-f7efa1246ef6 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:18:12.231529 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4808fa-768b-4c67-a51a-f7efa1246ef6 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:18:12.231808 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4808fa-768b-4c67-a51a-f7efa1246ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:18:12.252265 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-bf4808fa-768b-4c67-a51a-f7efa1246ef6 None None] MFA Rules not processed for user `9f2e208a67d4443eb00c75c60c54173f`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:18:12.253484 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-bf4808fa-768b-4c67-a51a-f7efa1246ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:18:12.316713 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1167/2320] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:18:12 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:18:15.010934 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=25 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:18:15.115974 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321e8f83-4976-494f-90fc-ae849b116381 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:18:15.117574 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-321e8f83-4976-494f-90fc-ae849b116381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:18:15.173510 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321e8f83-4976-494f-90fc-ae849b116381 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:18:15.177624 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002d6b27-161c-4e23-8bbb-3baa4502a992 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:18:15.178232 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d6b27-161c-4e23-8bbb-3baa4502a992 service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:18:15.178511 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d6b27-161c-4e23-8bbb-3baa4502a992 service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:18:15.178780 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d6b27-161c-4e23-8bbb-3baa4502a992 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:18:15.188535 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002d6b27-161c-4e23-8bbb-3baa4502a992 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:18:15.189977 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002d6b27-161c-4e23-8bbb-3baa4502a992 service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:18:15.202275 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1154/2321] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:18:15 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:18:22.308660 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=25 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:18:25.195958 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=26 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:18:29.633189 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4808fa-768b-4c67-a51a-f7efa1246ef6 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:18:29.647131 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4808fa-768b-4c67-a51a-f7efa1246ef6 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:18:29.652546 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce02a956-6730-4652-b864-c5f62e2aad50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae643cfdcb164aa9b474747cd21aa116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:18:29.653154 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02a956-6730-4652-b864-c5f62e2aad50 service nova] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:18:29.653962 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02a956-6730-4652-b864-c5f62e2aad50 service nova] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:18:29.654253 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02a956-6730-4652-b864-c5f62e2aad50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:18:29.656418 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-ce02a956-6730-4652-b864-c5f62e2aad50 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:18:29.667586 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce02a956-6730-4652-b864-c5f62e2aad50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:18:29.670085 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce02a956-6730-4652-b864-c5f62e2aad50 service nova] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:18:29.680060 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1168/2322] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:18:29 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:18:32.094390 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002d6b27-161c-4e23-8bbb-3baa4502a992 service neutron] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:18:32.103000 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002d6b27-161c-4e23-8bbb-3baa4502a992 service neutron] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:18:32.110811 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d1cbc2-7311-4789-837d-16a7abdc5ef4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:18:32.111497 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d1cbc2-7311-4789-837d-16a7abdc5ef4 service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:18:32.111923 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d1cbc2-7311-4789-837d-16a7abdc5ef4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:18:32.112324 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d1cbc2-7311-4789-837d-16a7abdc5ef4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:18:32.114287 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-73d1cbc2-7311-4789-837d-16a7abdc5ef4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:18:32.130388 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d1cbc2-7311-4789-837d-16a7abdc5ef4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:18:32.131875 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d1cbc2-7311-4789-837d-16a7abdc5ef4 service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:18:32.146251 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1155/2323] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:18:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:18:37.457135 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce02a956-6730-4652-b864-c5f62e2aad50 service nova] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:18:37.458640 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-ce02a956-6730-4652-b864-c5f62e2aad50 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:18:37.524061 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce02a956-6730-4652-b864-c5f62e2aad50 service nova] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:18:37.527913 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf182f4-4dde-430f-8bd2-f639af822ea3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:18:37.529195 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf182f4-4dde-430f-8bd2-f639af822ea3 service cinder] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:18:37.529539 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf182f4-4dde-430f-8bd2-f639af822ea3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:18:37.529879 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf182f4-4dde-430f-8bd2-f639af822ea3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:18:37.531681 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-bbf182f4-4dde-430f-8bd2-f639af822ea3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:18:37.547711 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf182f4-4dde-430f-8bd2-f639af822ea3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:18:37.549221 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf182f4-4dde-430f-8bd2-f639af822ea3 service cinder] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:18:37.561629 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1169/2324] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:18:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:18:42.140559 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=6 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:18:46.095249 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e442fb-e524-4bc0-99ca-64af528b09a7 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:18:46.095812 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e442fb-e524-4bc0-99ca-64af528b09a7 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:18:46.095812 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e442fb-e524-4bc0-99ca-64af528b09a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:18:46.115527 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-55e442fb-e524-4bc0-99ca-64af528b09a7 None None] MFA Rules not processed for user `de75fd7d7721433684cafa2e6d3faaae`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:18:46.116795 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-55e442fb-e524-4bc0-99ca-64af528b09a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:18:46.194882 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1156/2325] 10.222.0.42 () {58 vars in 967 bytes} [Thu Sep 8 01:18:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:18:46.253874 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf182f4-4dde-430f-8bd2-f639af822ea3 service cinder] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:18:46.257180 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-bbf182f4-4dde-430f-8bd2-f639af822ea3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:18:46.331772 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf182f4-4dde-430f-8bd2-f639af822ea3 service cinder] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:18:46.337217 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd71e335-70aa-4139-a331-6eb69ff0f15f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8add26872fe6479aa9d7de155a17195a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:18:46.337860 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd71e335-70aa-4139-a331-6eb69ff0f15f service swift] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:18:46.338135 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd71e335-70aa-4139-a331-6eb69ff0f15f service swift] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:18:46.338405 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd71e335-70aa-4139-a331-6eb69ff0f15f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:18:46.348075 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd71e335-70aa-4139-a331-6eb69ff0f15f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:18:46.349706 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd71e335-70aa-4139-a331-6eb69ff0f15f service swift] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:18:46.359160 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1170/2326] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Sep 8 01:18:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 01:18:47.992166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e442fb-e524-4bc0-99ca-64af528b09a7 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:18:47.993626 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-55e442fb-e524-4bc0-99ca-64af528b09a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:18:48.014959 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e442fb-e524-4bc0-99ca-64af528b09a7 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:18:48.018617 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3ac3e9-ad9d-443b-a9a6-59373fb00172 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:18:48.019220 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ac3e9-ad9d-443b-a9a6-59373fb00172 service cinder] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:18:48.019499 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ac3e9-ad9d-443b-a9a6-59373fb00172 service cinder] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:18:48.019768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ac3e9-ad9d-443b-a9a6-59373fb00172 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:18:48.021644 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-ef3ac3e9-ad9d-443b-a9a6-59373fb00172 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:18:48.033170 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3ac3e9-ad9d-443b-a9a6-59373fb00172 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:18:48.034720 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3ac3e9-ad9d-443b-a9a6-59373fb00172 service cinder] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:18:48.046549 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1157/2327] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:18:47 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:18:50.310195 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd71e335-70aa-4139-a331-6eb69ff0f15f service swift] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:18:50.340381 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd71e335-70aa-4139-a331-6eb69ff0f15f service swift] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:18:50.352483 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159c00ce-ebbb-498a-87a3-fb768891bc59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:18:50.353117 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159c00ce-ebbb-498a-87a3-fb768891bc59 service cinder] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:18:50.353570 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159c00ce-ebbb-498a-87a3-fb768891bc59 service cinder] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:18:50.354017 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159c00ce-ebbb-498a-87a3-fb768891bc59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:18:50.357836 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-159c00ce-ebbb-498a-87a3-fb768891bc59 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:18:50.464723 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159c00ce-ebbb-498a-87a3-fb768891bc59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:18:50.466803 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159c00ce-ebbb-498a-87a3-fb768891bc59 service cinder] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:18:50.486265 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1171/2328] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:18:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:18:52.555646 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de06e810-acd0-4aa2-a022-37147b34f58d None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:18:52.556365 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de06e810-acd0-4aa2-a022-37147b34f58d None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:18:52.556365 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de06e810-acd0-4aa2-a022-37147b34f58d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:18:52.577451 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-de06e810-acd0-4aa2-a022-37147b34f58d None None] MFA Rules not processed for user `83fea9762aaf4766a23f1774ff1411fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:18:52.578940 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-de06e810-acd0-4aa2-a022-37147b34f58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:18:52.661599 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1158/2329] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:18:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:18:53.187461 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159c00ce-ebbb-498a-87a3-fb768891bc59 service cinder] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:18:53.191894 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=93 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:18:53.198040 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159c00ce-ebbb-498a-87a3-fb768891bc59 service cinder] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:18:53.202010 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a33171-ddb4-4a34-bcf9-99d9da73fbff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:18:53.202640 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a33171-ddb4-4a34-bcf9-99d9da73fbff service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:18:53.202918 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a33171-ddb4-4a34-bcf9-99d9da73fbff service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:18:53.203195 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a33171-ddb4-4a34-bcf9-99d9da73fbff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:18:53.204995 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-67a33171-ddb4-4a34-bcf9-99d9da73fbff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:18:53.217699 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a33171-ddb4-4a34-bcf9-99d9da73fbff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:18:53.219332 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a33171-ddb4-4a34-bcf9-99d9da73fbff service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:18:53.228778 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1172/2330] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:18:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:18:53.637714 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de06e810-acd0-4aa2-a022-37147b34f58d None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:18:53.639531 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-de06e810-acd0-4aa2-a022-37147b34f58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:18:53.729541 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de06e810-acd0-4aa2-a022-37147b34f58d None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:18:53.735494 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a3b1d-57f9-4db1-b7f2-1b06aa1fb97b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '301d30bbd3de4cc69e96bb12729442c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:18:53.736272 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a3b1d-57f9-4db1-b7f2-1b06aa1fb97b service placement] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:18:53.736556 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a3b1d-57f9-4db1-b7f2-1b06aa1fb97b service placement] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:18:53.736828 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a3b1d-57f9-4db1-b7f2-1b06aa1fb97b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:18:53.738835 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-5d7a3b1d-57f9-4db1-b7f2-1b06aa1fb97b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:18:53.753721 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a3b1d-57f9-4db1-b7f2-1b06aa1fb97b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:18:53.755393 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a3b1d-57f9-4db1-b7f2-1b06aa1fb97b service placement] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:18:53.778772 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1159/2331] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:18:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:18:55.181227 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a33171-ddb4-4a34-bcf9-99d9da73fbff service neutron] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:18:55.182718 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-67a33171-ddb4-4a34-bcf9-99d9da73fbff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:18:55.261642 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a33171-ddb4-4a34-bcf9-99d9da73fbff service neutron] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:18:55.267220 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01b7521-ba52-4db4-9913-7e41cb011f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:18:55.267847 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b7521-ba52-4db4-9913-7e41cb011f2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:18:55.268126 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b7521-ba52-4db4-9913-7e41cb011f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:18:55.268425 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b7521-ba52-4db4-9913-7e41cb011f2f admin admin] PATH_INFO: `/v3/users/5411bb3267704943aea31c4bf1b4c9d1` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:18:55.269610 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01b7521-ba52-4db4-9913-7e41cb011f2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5411bb3267704943aea31c4bf1b4c9d1)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:18:55.271537 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01b7521-ba52-4db4-9913-7e41cb011f2f admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:18:55.319238 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-d01b7521-ba52-4db4-9913-7e41cb011f2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5411bb3267704943aea31c4bf1b4c9d1'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:18:55.331709 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-d01b7521-ba52-4db4-9913-7e41cb011f2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5411bb3267704943aea31c4bf1b4c9d1'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:18:55.338498 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-d01b7521-ba52-4db4-9913-7e41cb011f2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5411bb3267704943aea31c4bf1b4c9d1'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:18:55.347456 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-d01b7521-ba52-4db4-9913-7e41cb011f2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5411bb3267704943aea31c4bf1b4c9d1'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:18:55.348086 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-d01b7521-ba52-4db4-9913-7e41cb011f2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5411bb3267704943aea31c4bf1b4c9d1'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:18:55.354944 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1173/2332] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:18:55 2022] DELETE /identity/v3/users/5411bb3267704943aea31c4bf1b4c9d1 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:18:55.541227 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a3b1d-57f9-4db1-b7f2-1b06aa1fb97b service placement] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:18:55.554377 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a3b1d-57f9-4db1-b7f2-1b06aa1fb97b service placement] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:18:55.584881 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:18:55.624719 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62508fda-0dcb-49b7-abfe-c3911d786afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:18:55.625764 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62508fda-0dcb-49b7-abfe-c3911d786afc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:18:55.625764 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62508fda-0dcb-49b7-abfe-c3911d786afc admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:18:55.626005 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62508fda-0dcb-49b7-abfe-c3911d786afc admin admin] PATH_INFO: `/v3/projects/275f1c82d4f34cbfa81591e5f5a13f63` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:18:55.627040 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62508fda-0dcb-49b7-abfe-c3911d786afc admin admin] RBAC: Authorizing `identity:delete_project(project_id=275f1c82d4f34cbfa81591e5f5a13f63)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:18:55.629070 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62508fda-0dcb-49b7-abfe-c3911d786afc admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:18:55.678539 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-62508fda-0dcb-49b7-abfe-c3911d786afc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '275f1c82d4f34cbfa81591e5f5a13f63', 'request_id': 'req-62508fda-0dcb-49b7-abfe-c3911d786afc'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:18:55.686925 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-62508fda-0dcb-49b7-abfe-c3911d786afc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '275f1c82d4f34cbfa81591e5f5a13f63', 'request_id': 'req-62508fda-0dcb-49b7-abfe-c3911d786afc'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:18:55.693837 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-62508fda-0dcb-49b7-abfe-c3911d786afc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '275f1c82d4f34cbfa81591e5f5a13f63', 'request_id': 'req-62508fda-0dcb-49b7-abfe-c3911d786afc'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:18:55.706924 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-62508fda-0dcb-49b7-abfe-c3911d786afc admin admin] The token cache is being invalidate because project 275f1c82d4f34cbfa81591e5f5a13f63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81661) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:18:55.707212 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-62508fda-0dcb-49b7-abfe-c3911d786afc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:18:55.707860 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-62508fda-0dcb-49b7-abfe-c3911d786afc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:18:55.709692 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1160/2333] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:18:55 2022] DELETE /identity/v3/projects/275f1c82d4f34cbfa81591e5f5a13f63 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:18:55.713839 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01b7521-ba52-4db4-9913-7e41cb011f2f admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:18:55.715736 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-d01b7521-ba52-4db4-9913-7e41cb011f2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:18:55.729957 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01b7521-ba52-4db4-9913-7e41cb011f2f admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:18:55.793549 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48214d40-f051-4e9d-9a3f-01497b79339f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:18:55.794381 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48214d40-f051-4e9d-9a3f-01497b79339f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:18:55.794661 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48214d40-f051-4e9d-9a3f-01497b79339f admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:18:55.794936 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48214d40-f051-4e9d-9a3f-01497b79339f admin admin] PATH_INFO: `/v3/users/dc29e7a08e1d4f0fbd259ee49b4a97d0` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:18:55.796067 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48214d40-f051-4e9d-9a3f-01497b79339f admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc29e7a08e1d4f0fbd259ee49b4a97d0)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:18:55.798191 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48214d40-f051-4e9d-9a3f-01497b79339f admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:18:55.809366 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:18:55.862026 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-48214d40-f051-4e9d-9a3f-01497b79339f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc29e7a08e1d4f0fbd259ee49b4a97d0'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:18:55.876679 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-48214d40-f051-4e9d-9a3f-01497b79339f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc29e7a08e1d4f0fbd259ee49b4a97d0'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:18:55.896842 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-48214d40-f051-4e9d-9a3f-01497b79339f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc29e7a08e1d4f0fbd259ee49b4a97d0'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:18:55.909689 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-48214d40-f051-4e9d-9a3f-01497b79339f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc29e7a08e1d4f0fbd259ee49b4a97d0'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:18:55.910267 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-48214d40-f051-4e9d-9a3f-01497b79339f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc29e7a08e1d4f0fbd259ee49b4a97d0'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:18:55.919926 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1174/2334] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:18:55 2022] DELETE /identity/v3/users/dc29e7a08e1d4f0fbd259ee49b4a97d0 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:18:56.105015 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62508fda-0dcb-49b7-abfe-c3911d786afc admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:18:56.106495 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-62508fda-0dcb-49b7-abfe-c3911d786afc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:18:56.119272 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62508fda-0dcb-49b7-abfe-c3911d786afc admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:18:56.197166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b67677c-0b89-49c8-b067-c76965dde98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:18:56.197805 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b67677c-0b89-49c8-b067-c76965dde98c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:18:56.198092 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b67677c-0b89-49c8-b067-c76965dde98c admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:18:56.198381 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b67677c-0b89-49c8-b067-c76965dde98c admin admin] PATH_INFO: `/v3/projects/07b8c4b08bb041169bb932e2de46711a` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:18:56.200100 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b67677c-0b89-49c8-b067-c76965dde98c admin admin] RBAC: Authorizing `identity:delete_project(project_id=07b8c4b08bb041169bb932e2de46711a)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:18:56.202200 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b67677c-0b89-49c8-b067-c76965dde98c admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:18:56.254920 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-4b67677c-0b89-49c8-b067-c76965dde98c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07b8c4b08bb041169bb932e2de46711a', 'request_id': 'req-4b67677c-0b89-49c8-b067-c76965dde98c'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:18:56.265692 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-4b67677c-0b89-49c8-b067-c76965dde98c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07b8c4b08bb041169bb932e2de46711a', 'request_id': 'req-4b67677c-0b89-49c8-b067-c76965dde98c'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:18:56.269316 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:18:56.276460 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-4b67677c-0b89-49c8-b067-c76965dde98c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07b8c4b08bb041169bb932e2de46711a', 'request_id': 'req-4b67677c-0b89-49c8-b067-c76965dde98c'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:18:56.292081 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-4b67677c-0b89-49c8-b067-c76965dde98c admin admin] The token cache is being invalidate because project 07b8c4b08bb041169bb932e2de46711a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81661) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:18:56.292392 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-4b67677c-0b89-49c8-b067-c76965dde98c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:18:56.294421 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-4b67677c-0b89-49c8-b067-c76965dde98c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:18:56.296255 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1161/2335] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:18:56 2022] DELETE /identity/v3/projects/07b8c4b08bb041169bb932e2de46711a => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:18:59.712476 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48214d40-f051-4e9d-9a3f-01497b79339f admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:18:59.714088 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-48214d40-f051-4e9d-9a3f-01497b79339f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:18:59.858694 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48214d40-f051-4e9d-9a3f-01497b79339f admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:18:59.875562 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b67677c-0b89-49c8-b067-c76965dde98c admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:18:59.877435 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-4b67677c-0b89-49c8-b067-c76965dde98c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:18:59.994633 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d4346f-fb3b-446c-b40c-eaab21dcec55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:00.012997 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d4346f-fb3b-446c-b40c-eaab21dcec55 service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:00.013308 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d4346f-fb3b-446c-b40c-eaab21dcec55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:00.018241 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d4346f-fb3b-446c-b40c-eaab21dcec55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:00.024328 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-80d4346f-fb3b-446c-b40c-eaab21dcec55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:00.070370 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d4346f-fb3b-446c-b40c-eaab21dcec55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:00.071536 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d4346f-fb3b-446c-b40c-eaab21dcec55 service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:00.112426 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1175/2336] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:18:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:19:00.141161 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b67677c-0b89-49c8-b067-c76965dde98c admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:00.147289 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c26423-10d9-453e-8e32-336d3f9d84db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae643cfdcb164aa9b474747cd21aa116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:00.147927 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c26423-10d9-453e-8e32-336d3f9d84db service nova] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:00.148242 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c26423-10d9-453e-8e32-336d3f9d84db service nova] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:00.148522 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c26423-10d9-453e-8e32-336d3f9d84db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:00.150377 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-96c26423-10d9-453e-8e32-336d3f9d84db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:00.175836 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c26423-10d9-453e-8e32-336d3f9d84db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:00.180212 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c26423-10d9-453e-8e32-336d3f9d84db service nova] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:00.207270 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1162/2337] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:18:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:19:00.999330 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d4346f-fb3b-446c-b40c-eaab21dcec55 service neutron] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:01.001010 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-80d4346f-fb3b-446c-b40c-eaab21dcec55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:01.025523 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d4346f-fb3b-446c-b40c-eaab21dcec55 service neutron] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:01.032566 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3f7aad-2b04-4852-87a9-e8466157a5cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae643cfdcb164aa9b474747cd21aa116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:01.033272 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3f7aad-2b04-4852-87a9-e8466157a5cb service nova] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:01.033752 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3f7aad-2b04-4852-87a9-e8466157a5cb service nova] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:01.034221 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3f7aad-2b04-4852-87a9-e8466157a5cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:01.036061 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-aa3f7aad-2b04-4852-87a9-e8466157a5cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:01.052089 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3f7aad-2b04-4852-87a9-e8466157a5cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:01.054070 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3f7aad-2b04-4852-87a9-e8466157a5cb service nova] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:01.066097 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1176/2338] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:19:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:19:06.864526 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff027307-8199-4c68-96dd-624470871ce4 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:06.865149 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff027307-8199-4c68-96dd-624470871ce4 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:06.865149 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff027307-8199-4c68-96dd-624470871ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:06.885590 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-ff027307-8199-4c68-96dd-624470871ce4 None None] MFA Rules not processed for user `3246a8dea80d4d5b8c3fb50d5f1f31c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:19:06.886936 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-ff027307-8199-4c68-96dd-624470871ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:06.943932 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1163/2339] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:19:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:19:11.060938 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:DELETE=10,keystone:SELECT=74,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:19:11.368504 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3f7aad-2b04-4852-87a9-e8466157a5cb service nova] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:11.409713 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3f7aad-2b04-4852-87a9-e8466157a5cb service nova] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:11.426646 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80962f4a-1a6d-49b9-8699-64c0f70f2eab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae643cfdcb164aa9b474747cd21aa116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:11.430097 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80962f4a-1a6d-49b9-8699-64c0f70f2eab service nova] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:11.430471 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80962f4a-1a6d-49b9-8699-64c0f70f2eab service nova] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:11.430960 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80962f4a-1a6d-49b9-8699-64c0f70f2eab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:11.436448 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-80962f4a-1a6d-49b9-8699-64c0f70f2eab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:11.475188 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80962f4a-1a6d-49b9-8699-64c0f70f2eab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:11.477579 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80962f4a-1a6d-49b9-8699-64c0f70f2eab service nova] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:11.502273 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1177/2340] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:19:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:19:14.189790 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff027307-8199-4c68-96dd-624470871ce4 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:14.191364 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-ff027307-8199-4c68-96dd-624470871ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:14.272643 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff027307-8199-4c68-96dd-624470871ce4 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:14.279396 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd7bb30-1595-4d97-b5f5-3d9d34affbf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:14.280182 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd7bb30-1595-4d97-b5f5-3d9d34affbf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:14.280374 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd7bb30-1595-4d97-b5f5-3d9d34affbf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:14.280822 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd7bb30-1595-4d97-b5f5-3d9d34affbf7 admin admin] PATH_INFO: `/v3/users/336e7f07cd7d461c8257995a772ac1f0` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:14.282224 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd7bb30-1595-4d97-b5f5-3d9d34affbf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=336e7f07cd7d461c8257995a772ac1f0)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:14.284676 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd7bb30-1595-4d97-b5f5-3d9d34affbf7 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:14.311950 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:19:14.344021 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-9cd7bb30-1595-4d97-b5f5-3d9d34affbf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '336e7f07cd7d461c8257995a772ac1f0'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:14.357340 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-9cd7bb30-1595-4d97-b5f5-3d9d34affbf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '336e7f07cd7d461c8257995a772ac1f0'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:14.365390 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-9cd7bb30-1595-4d97-b5f5-3d9d34affbf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '336e7f07cd7d461c8257995a772ac1f0'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:14.366160 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-9cd7bb30-1595-4d97-b5f5-3d9d34affbf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '336e7f07cd7d461c8257995a772ac1f0'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:14.375898 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-9cd7bb30-1595-4d97-b5f5-3d9d34affbf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '336e7f07cd7d461c8257995a772ac1f0'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:14.382187 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1164/2341] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:19:14 2022] DELETE /identity/v3/users/336e7f07cd7d461c8257995a772ac1f0 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:19:14.664735 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80962f4a-1a6d-49b9-8699-64c0f70f2eab service nova] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:14.682588 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80962f4a-1a6d-49b9-8699-64c0f70f2eab service nova] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:14.743550 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b24e8a4-095f-4da3-8c1b-3ccb889313b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:14.744222 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b24e8a4-095f-4da3-8c1b-3ccb889313b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:14.744510 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b24e8a4-095f-4da3-8c1b-3ccb889313b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:14.744789 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b24e8a4-095f-4da3-8c1b-3ccb889313b0 admin admin] PATH_INFO: `/v3/projects/93477e5edb38498ba32feb52519ba584` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:14.745951 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b24e8a4-095f-4da3-8c1b-3ccb889313b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93477e5edb38498ba32feb52519ba584)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:14.747875 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b24e8a4-095f-4da3-8c1b-3ccb889313b0 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:14.799164 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-1b24e8a4-095f-4da3-8c1b-3ccb889313b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93477e5edb38498ba32feb52519ba584', 'request_id': 'req-1b24e8a4-095f-4da3-8c1b-3ccb889313b0'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:14.805445 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-1b24e8a4-095f-4da3-8c1b-3ccb889313b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93477e5edb38498ba32feb52519ba584', 'request_id': 'req-1b24e8a4-095f-4da3-8c1b-3ccb889313b0'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:14.811383 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-1b24e8a4-095f-4da3-8c1b-3ccb889313b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93477e5edb38498ba32feb52519ba584', 'request_id': 'req-1b24e8a4-095f-4da3-8c1b-3ccb889313b0'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:14.826354 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-1b24e8a4-095f-4da3-8c1b-3ccb889313b0 admin admin] The token cache is being invalidate because project 93477e5edb38498ba32feb52519ba584 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:19:14.827050 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-1b24e8a4-095f-4da3-8c1b-3ccb889313b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:14.827704 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-1b24e8a4-095f-4da3-8c1b-3ccb889313b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:14.829380 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1178/2342] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:19:14 2022] DELETE /identity/v3/projects/93477e5edb38498ba32feb52519ba584 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:19:14.834150 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece37ba5-3b14-4742-abef-6880f2186961 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:14.834438 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece37ba5-3b14-4742-abef-6880f2186961 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:14.834712 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece37ba5-3b14-4742-abef-6880f2186961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:14.854751 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-ece37ba5-3b14-4742-abef-6880f2186961 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:19:14.855906 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-ece37ba5-3b14-4742-abef-6880f2186961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:14.917122 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1165/2343] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:19:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:19:14.925277 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d9bcd-f6cd-4987-a525-eea614e9b168 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:14.925703 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d9bcd-f6cd-4987-a525-eea614e9b168 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:14.926016 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d9bcd-f6cd-4987-a525-eea614e9b168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:14.945750 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-bc1d9bcd-f6cd-4987-a525-eea614e9b168 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:19:14.946971 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-bc1d9bcd-f6cd-4987-a525-eea614e9b168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:14.955898 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1179/2344] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:19:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:19:14.959962 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece37ba5-3b14-4742-abef-6880f2186961 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:14.970570 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece37ba5-3b14-4742-abef-6880f2186961 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:14.975271 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4569a13-89ef-4b5d-be41-41bc5e1be53c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:14.975878 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4569a13-89ef-4b5d-be41-41bc5e1be53c admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:14.976233 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4569a13-89ef-4b5d-be41-41bc5e1be53c admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:14.976549 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4569a13-89ef-4b5d-be41-41bc5e1be53c admin admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:14.977513 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4569a13-89ef-4b5d-be41-41bc5e1be53c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:19:14.978017 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4569a13-89ef-4b5d-be41-41bc5e1be53c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:14.980388 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4569a13-89ef-4b5d-be41-41bc5e1be53c admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:14.996247 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1166/2345] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:19:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:19:14.997549 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d9bcd-f6cd-4987-a525-eea614e9b168 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:15.006972 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d9bcd-f6cd-4987-a525-eea614e9b168 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:15.012813 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2d4594-b94d-4bc0-9f77-f831e23c84df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:15.013446 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d4594-b94d-4bc0-9f77-f831e23c84df admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:15.013751 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d4594-b94d-4bc0-9f77-f831e23c84df admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:15.014024 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d4594-b94d-4bc0-9f77-f831e23c84df admin admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:15.014661 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2d4594-b94d-4bc0-9f77-f831e23c84df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:15.016624 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2d4594-b94d-4bc0-9f77-f831e23c84df admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:15.050429 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1180/2346] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:19:14 2022] POST /identity/v3/projects => generated 389 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:19:15.051716 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4569a13-89ef-4b5d-be41-41bc5e1be53c admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:15.061755 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4569a13-89ef-4b5d-be41-41bc5e1be53c admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:15.118960 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23fb498-2303-41c9-9153-c64ab0827859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:15.119479 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23fb498-2303-41c9-9153-c64ab0827859 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:15.119867 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23fb498-2303-41c9-9153-c64ab0827859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:15.120188 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23fb498-2303-41c9-9153-c64ab0827859 admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:15.121741 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23fb498-2303-41c9-9153-c64ab0827859 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:15.123674 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23fb498-2303-41c9-9153-c64ab0827859 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:15.146326 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1167/2347] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:19:15 2022] POST /identity/v3/users => generated 295 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:19:15.150428 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2d4594-b94d-4bc0-9f77-f831e23c84df admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:15.161082 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2d4594-b94d-4bc0-9f77-f831e23c84df admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:15.165696 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0bb54-6a74-472f-9b4f-1d42edc6e703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:15.166288 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0bb54-6a74-472f-9b4f-1d42edc6e703 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:15.166700 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0bb54-6a74-472f-9b4f-1d42edc6e703 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:15.166968 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0bb54-6a74-472f-9b4f-1d42edc6e703 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:15.167657 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf0bb54-6a74-472f-9b4f-1d42edc6e703 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:15.169310 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf0bb54-6a74-472f-9b4f-1d42edc6e703 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:15.179244 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1181/2348] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 01:19:15.183268 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23fb498-2303-41c9-9153-c64ab0827859 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:15.194171 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23fb498-2303-41c9-9153-c64ab0827859 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:15.199645 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16683bd-6b14-4690-9a49-71356a9785db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:15.200370 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16683bd-6b14-4690-9a49-71356a9785db admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:15.200770 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16683bd-6b14-4690-9a49-71356a9785db admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:15.201108 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16683bd-6b14-4690-9a49-71356a9785db admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:15.201910 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16683bd-6b14-4690-9a49-71356a9785db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:15.203599 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16683bd-6b14-4690-9a49-71356a9785db admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:15.215790 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1168/2349] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:19:15.220518 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0bb54-6a74-472f-9b4f-1d42edc6e703 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:15.231748 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0bb54-6a74-472f-9b4f-1d42edc6e703 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:15.236556 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06351324-87be-4d8f-a653-43fc68cc99c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:15.237206 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06351324-87be-4d8f-a653-43fc68cc99c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:15.237489 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06351324-87be-4d8f-a653-43fc68cc99c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:15.237762 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06351324-87be-4d8f-a653-43fc68cc99c9 admin admin] PATH_INFO: `/v3/projects/1fd71ac60f9a4506a9eda527bcce28c8/users/a95cb912b58e4e0f9225c6c0511d9105/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:15.255146 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06351324-87be-4d8f-a653-43fc68cc99c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fd71ac60f9a4506a9eda527bcce28c8, user_id=a95cb912b58e4e0f9225c6c0511d9105, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:15.257867 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06351324-87be-4d8f-a653-43fc68cc99c9 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:15.273087 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1182/2350] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:19:15 2022] PUT /identity/v3/projects/1fd71ac60f9a4506a9eda527bcce28c8/users/a95cb912b58e4e0f9225c6c0511d9105/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:19:15.285068 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace93d49-27d5-4c78-be66-7ad6acb0325c None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:15.285351 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace93d49-27d5-4c78-be66-7ad6acb0325c None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:15.285631 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace93d49-27d5-4c78-be66-7ad6acb0325c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:15.294305 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:19:15.320225 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-ace93d49-27d5-4c78-be66-7ad6acb0325c None None] MFA Rules not processed for user `a95cb912b58e4e0f9225c6c0511d9105`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:19:15.321444 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-ace93d49-27d5-4c78-be66-7ad6acb0325c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:15.395146 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1169/2351] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:19:15 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:19:15.408766 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06351324-87be-4d8f-a653-43fc68cc99c9 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:15.410754 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-06351324-87be-4d8f-a653-43fc68cc99c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:15.428416 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:19:15.494272 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06351324-87be-4d8f-a653-43fc68cc99c9 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:15.502112 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb1cee0-bb2d-4844-bc86-35443a25d628 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:15.502835 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb1cee0-bb2d-4844-bc86-35443a25d628 service cinder] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:15.503297 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb1cee0-bb2d-4844-bc86-35443a25d628 service cinder] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:15.503668 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb1cee0-bb2d-4844-bc86-35443a25d628 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:15.515952 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb1cee0-bb2d-4844-bc86-35443a25d628 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:15.517876 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb1cee0-bb2d-4844-bc86-35443a25d628 service cinder] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:15.532055 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1183/2352] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:19:15 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:19:24.822254 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace93d49-27d5-4c78-be66-7ad6acb0325c None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:24.824370 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-ace93d49-27d5-4c78-be66-7ad6acb0325c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:24.882933 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace93d49-27d5-4c78-be66-7ad6acb0325c None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:24.887771 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e013586-f8f4-4d30-bf1d-049a28d4181d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:24.888412 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e013586-f8f4-4d30-bf1d-049a28d4181d service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:24.888912 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e013586-f8f4-4d30-bf1d-049a28d4181d service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:24.889029 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e013586-f8f4-4d30-bf1d-049a28d4181d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:24.891089 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-5e013586-f8f4-4d30-bf1d-049a28d4181d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:24.961351 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e013586-f8f4-4d30-bf1d-049a28d4181d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:24.962712 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e013586-f8f4-4d30-bf1d-049a28d4181d service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:24.972584 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1170/2353] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:19:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:19:25.525515 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=20 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:19:28.512618 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb1cee0-bb2d-4844-bc86-35443a25d628 service cinder] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:28.514106 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-cbb1cee0-bb2d-4844-bc86-35443a25d628 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:28.531651 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb1cee0-bb2d-4844-bc86-35443a25d628 service cinder] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:28.536544 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19115459-b3a7-48de-9bbf-e4a9955f4120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:28.537400 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19115459-b3a7-48de-9bbf-e4a9955f4120 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:28.537830 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19115459-b3a7-48de-9bbf-e4a9955f4120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:28.538110 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19115459-b3a7-48de-9bbf-e4a9955f4120 admin admin] PATH_INFO: `/v3/users/20af4425b7434d2dab7baeacbb845387` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:28.539327 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19115459-b3a7-48de-9bbf-e4a9955f4120 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20af4425b7434d2dab7baeacbb845387)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:28.541285 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19115459-b3a7-48de-9bbf-e4a9955f4120 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:28.596707 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-19115459-b3a7-48de-9bbf-e4a9955f4120 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20af4425b7434d2dab7baeacbb845387'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:28.610838 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-19115459-b3a7-48de-9bbf-e4a9955f4120 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20af4425b7434d2dab7baeacbb845387'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:28.618222 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-19115459-b3a7-48de-9bbf-e4a9955f4120 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20af4425b7434d2dab7baeacbb845387'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:28.629598 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-19115459-b3a7-48de-9bbf-e4a9955f4120 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20af4425b7434d2dab7baeacbb845387'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:28.630224 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-19115459-b3a7-48de-9bbf-e4a9955f4120 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20af4425b7434d2dab7baeacbb845387'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:28.637899 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1184/2354] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:19:28 2022] DELETE /identity/v3/users/20af4425b7434d2dab7baeacbb845387 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:19:28.811566 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e013586-f8f4-4d30-bf1d-049a28d4181d service neutron] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:28.821548 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e013586-f8f4-4d30-bf1d-049a28d4181d service neutron] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:28.863438 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:19:28.881607 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eb2b3e-5990-497b-894c-f41dbc1e2fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:28.882227 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb2b3e-5990-497b-894c-f41dbc1e2fc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:28.882499 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb2b3e-5990-497b-894c-f41dbc1e2fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:28.882766 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb2b3e-5990-497b-894c-f41dbc1e2fc8 admin admin] PATH_INFO: `/v3/projects/f520f61ec5304a72ac707318d3a73b4f` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:28.884282 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eb2b3e-5990-497b-894c-f41dbc1e2fc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f520f61ec5304a72ac707318d3a73b4f)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:28.886247 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eb2b3e-5990-497b-894c-f41dbc1e2fc8 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:28.939102 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-37eb2b3e-5990-497b-894c-f41dbc1e2fc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f520f61ec5304a72ac707318d3a73b4f', 'request_id': 'req-37eb2b3e-5990-497b-894c-f41dbc1e2fc8'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:28.948042 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-37eb2b3e-5990-497b-894c-f41dbc1e2fc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f520f61ec5304a72ac707318d3a73b4f', 'request_id': 'req-37eb2b3e-5990-497b-894c-f41dbc1e2fc8'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:28.955385 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-37eb2b3e-5990-497b-894c-f41dbc1e2fc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f520f61ec5304a72ac707318d3a73b4f', 'request_id': 'req-37eb2b3e-5990-497b-894c-f41dbc1e2fc8'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:28.972318 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-37eb2b3e-5990-497b-894c-f41dbc1e2fc8 admin admin] The token cache is being invalidate because project f520f61ec5304a72ac707318d3a73b4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81661) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:19:28.972318 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-37eb2b3e-5990-497b-894c-f41dbc1e2fc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:28.972318 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-37eb2b3e-5990-497b-894c-f41dbc1e2fc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:28.973528 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1171/2355] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:19:28 2022] DELETE /identity/v3/projects/f520f61ec5304a72ac707318d3a73b4f => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:19:28.984804 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87991e37-1572-4bfa-8946-df3924dd11d5 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:28.985086 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87991e37-1572-4bfa-8946-df3924dd11d5 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:28.985363 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87991e37-1572-4bfa-8946-df3924dd11d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:29.006358 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-87991e37-1572-4bfa-8946-df3924dd11d5 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:19:29.007624 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-87991e37-1572-4bfa-8946-df3924dd11d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:29.084374 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1185/2356] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:19:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:19:29.088260 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5af777-1800-43e4-ac9a-3db239cea410 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:29.088613 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5af777-1800-43e4-ac9a-3db239cea410 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:29.088879 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5af777-1800-43e4-ac9a-3db239cea410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:29.111633 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-7b5af777-1800-43e4-ac9a-3db239cea410 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:19:29.112898 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-7b5af777-1800-43e4-ac9a-3db239cea410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:29.122010 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1172/2357] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:19:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:19:29.126221 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87991e37-1572-4bfa-8946-df3924dd11d5 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:29.137433 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87991e37-1572-4bfa-8946-df3924dd11d5 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:29.142295 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee85686-5886-4d9b-b69a-4f178e48cf2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:29.143038 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee85686-5886-4d9b-b69a-4f178e48cf2a admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:29.143317 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee85686-5886-4d9b-b69a-4f178e48cf2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:29.143648 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee85686-5886-4d9b-b69a-4f178e48cf2a admin admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:29.144543 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee85686-5886-4d9b-b69a-4f178e48cf2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:19:29.144852 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee85686-5886-4d9b-b69a-4f178e48cf2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:29.146452 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee85686-5886-4d9b-b69a-4f178e48cf2a admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:29.160997 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1186/2358] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:19:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:19:29.165723 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5af777-1800-43e4-ac9a-3db239cea410 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:29.176936 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5af777-1800-43e4-ac9a-3db239cea410 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:29.181808 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b5664e-6080-49b7-b433-0696de88543c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:29.182418 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5664e-6080-49b7-b433-0696de88543c admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:29.182893 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5664e-6080-49b7-b433-0696de88543c admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:29.183272 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5664e-6080-49b7-b433-0696de88543c admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:29.183845 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b5664e-6080-49b7-b433-0696de88543c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:29.185905 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b5664e-6080-49b7-b433-0696de88543c admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:29.224526 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1173/2359] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:19:29 2022] POST /identity/v3/projects => generated 405 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:19:29.225631 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee85686-5886-4d9b-b69a-4f178e48cf2a admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:29.236763 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee85686-5886-4d9b-b69a-4f178e48cf2a admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:29.302135 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d1e67d-3614-4ab2-8e5f-287cfae021c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:29.302743 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d1e67d-3614-4ab2-8e5f-287cfae021c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:29.303099 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d1e67d-3614-4ab2-8e5f-287cfae021c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:29.303375 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d1e67d-3614-4ab2-8e5f-287cfae021c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:29.304004 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d1e67d-3614-4ab2-8e5f-287cfae021c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:29.306027 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d1e67d-3614-4ab2-8e5f-287cfae021c0 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:29.314658 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:19:29.333466 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1187/2360] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:19:29 2022] POST /identity/v3/users => generated 303 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:19:29.337497 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b5664e-6080-49b7-b433-0696de88543c admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:29.349245 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b5664e-6080-49b7-b433-0696de88543c admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:29.353888 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7290c5ff-41ed-41f2-93c0-f987b48e07da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:29.355506 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7290c5ff-41ed-41f2-93c0-f987b48e07da admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:29.355792 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7290c5ff-41ed-41f2-93c0-f987b48e07da admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:29.356063 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7290c5ff-41ed-41f2-93c0-f987b48e07da admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:29.356735 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7290c5ff-41ed-41f2-93c0-f987b48e07da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:29.358540 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7290c5ff-41ed-41f2-93c0-f987b48e07da admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:29.374846 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1174/2361] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:19:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 01:19:29.378301 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d1e67d-3614-4ab2-8e5f-287cfae021c0 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:29.389989 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d1e67d-3614-4ab2-8e5f-287cfae021c0 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:29.394999 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f763167-d7b4-4976-87d3-b10e9909ef04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:29.395614 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f763167-d7b4-4976-87d3-b10e9909ef04 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:29.395898 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f763167-d7b4-4976-87d3-b10e9909ef04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:29.397078 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f763167-d7b4-4976-87d3-b10e9909ef04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:29.397735 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f763167-d7b4-4976-87d3-b10e9909ef04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:29.399342 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f763167-d7b4-4976-87d3-b10e9909ef04 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:29.412439 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1188/2362] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:19:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 01:19:29.419346 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7290c5ff-41ed-41f2-93c0-f987b48e07da admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:29.429526 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7290c5ff-41ed-41f2-93c0-f987b48e07da admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:29.435143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed858faa-52c7-4947-9f78-49144b33cf82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:29.435830 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed858faa-52c7-4947-9f78-49144b33cf82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:29.436114 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed858faa-52c7-4947-9f78-49144b33cf82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:29.436415 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed858faa-52c7-4947-9f78-49144b33cf82 admin admin] PATH_INFO: `/v3/projects/35752e1b1e3a47e489a160378c591e03/users/0cd24f610d9f4e498e6a7d27185ec02b/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:29.453438 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed858faa-52c7-4947-9f78-49144b33cf82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35752e1b1e3a47e489a160378c591e03, user_id=0cd24f610d9f4e498e6a7d27185ec02b, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:29.457322 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed858faa-52c7-4947-9f78-49144b33cf82 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:29.473731 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1175/2363] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:19:29 2022] PUT /identity/v3/projects/35752e1b1e3a47e489a160378c591e03/users/0cd24f610d9f4e498e6a7d27185ec02b/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:19:29.479117 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad5d2c-8cc5-41c4-b8aa-fe9dba890980 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:29.479406 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad5d2c-8cc5-41c4-b8aa-fe9dba890980 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:29.479711 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad5d2c-8cc5-41c4-b8aa-fe9dba890980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:29.502805 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-42ad5d2c-8cc5-41c4-b8aa-fe9dba890980 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:19:29.504164 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-42ad5d2c-8cc5-41c4-b8aa-fe9dba890980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:29.573083 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1189/2364] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:19:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:19:29.584707 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed858faa-52c7-4947-9f78-49144b33cf82 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:29.587290 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-ed858faa-52c7-4947-9f78-49144b33cf82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:29.663632 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed858faa-52c7-4947-9f78-49144b33cf82 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:29.668036 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ff9610-fb18-4815-8e7d-f402f00edefc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:29.668809 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff9610-fb18-4815-8e7d-f402f00edefc service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:29.669141 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff9610-fb18-4815-8e7d-f402f00edefc service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:29.669419 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff9610-fb18-4815-8e7d-f402f00edefc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:29.680010 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ff9610-fb18-4815-8e7d-f402f00edefc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:29.681658 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ff9610-fb18-4815-8e7d-f402f00edefc service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:29.694608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1176/2365] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:19:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:19:30.403907 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3111b5-38af-429d-9a38-f8790ad0d3e8 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:30.404296 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3111b5-38af-429d-9a38-f8790ad0d3e8 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:30.404614 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3111b5-38af-429d-9a38-f8790ad0d3e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:30.441843 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-9f3111b5-38af-429d-9a38-f8790ad0d3e8 None None] MFA Rules not processed for user `0cd24f610d9f4e498e6a7d27185ec02b`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:19:30.443066 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-9f3111b5-38af-429d-9a38-f8790ad0d3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:30.523838 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1190/2366] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:19:30 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:19:30.525438 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ff9610-fb18-4815-8e7d-f402f00edefc service neutron] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:30.537434 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ff9610-fb18-4815-8e7d-f402f00edefc service neutron] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:30.542281 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e2228d-8202-423b-bfb0-9a7cedab7896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:30.542883 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2228d-8202-423b-bfb0-9a7cedab7896 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:30.543164 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2228d-8202-423b-bfb0-9a7cedab7896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:30.543434 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2228d-8202-423b-bfb0-9a7cedab7896 admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:30.544065 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e2228d-8202-423b-bfb0-9a7cedab7896 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:30.546027 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e2228d-8202-423b-bfb0-9a7cedab7896 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:30.555722 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:19:30.581407 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1177/2367] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:19:30 2022] POST /identity/v3/projects => generated 403 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:19:30.585562 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3111b5-38af-429d-9a38-f8790ad0d3e8 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:30.599653 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3111b5-38af-429d-9a38-f8790ad0d3e8 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:30.660125 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2efca0b-c1ef-482a-8b16-9a439be9f946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:30.660774 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2efca0b-c1ef-482a-8b16-9a439be9f946 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:30.661047 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2efca0b-c1ef-482a-8b16-9a439be9f946 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:30.661326 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2efca0b-c1ef-482a-8b16-9a439be9f946 admin admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:30.661976 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2efca0b-c1ef-482a-8b16-9a439be9f946 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:30.663938 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2efca0b-c1ef-482a-8b16-9a439be9f946 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:30.679322 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:19:30.688461 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1191/2368] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:19:30 2022] POST /identity/v3/users => generated 308 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:19:30.690674 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e2228d-8202-423b-bfb0-9a7cedab7896 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:30.702170 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e2228d-8202-423b-bfb0-9a7cedab7896 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:30.706918 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2d1db9-62e3-4150-99a3-e035657e569d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:30.707521 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d1db9-62e3-4150-99a3-e035657e569d admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:30.707802 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d1db9-62e3-4150-99a3-e035657e569d admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:30.708078 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d1db9-62e3-4150-99a3-e035657e569d admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:30.708746 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2d1db9-62e3-4150-99a3-e035657e569d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:30.710349 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2d1db9-62e3-4150-99a3-e035657e569d admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:30.720956 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1178/2369] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:19:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:19:30.725150 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2efca0b-c1ef-482a-8b16-9a439be9f946 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:30.735179 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2efca0b-c1ef-482a-8b16-9a439be9f946 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:30.740403 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122654d9-20b7-4023-9e03-7e92af30ab17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:30.741065 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122654d9-20b7-4023-9e03-7e92af30ab17 admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:30.741424 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122654d9-20b7-4023-9e03-7e92af30ab17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:30.741764 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122654d9-20b7-4023-9e03-7e92af30ab17 admin admin] PATH_INFO: `/v3/projects/b6c4982d371844ed8eb589552e65cd85/users/d02424de72204eb0a78d62d688588325/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:30.758786 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122654d9-20b7-4023-9e03-7e92af30ab17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6c4982d371844ed8eb589552e65cd85, user_id=d02424de72204eb0a78d62d688588325, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:30.762073 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122654d9-20b7-4023-9e03-7e92af30ab17 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:30.776354 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1192/2370] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:19:30 2022] PUT /identity/v3/projects/b6c4982d371844ed8eb589552e65cd85/users/d02424de72204eb0a78d62d688588325/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:19:30.781037 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2d1db9-62e3-4150-99a3-e035657e569d admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:30.791174 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2d1db9-62e3-4150-99a3-e035657e569d admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:30.854811 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a87b9e5-8e58-402e-8ee1-b83130217673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:30.855547 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87b9e5-8e58-402e-8ee1-b83130217673 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:30.855776 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87b9e5-8e58-402e-8ee1-b83130217673 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:30.856051 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87b9e5-8e58-402e-8ee1-b83130217673 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:30.856718 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a87b9e5-8e58-402e-8ee1-b83130217673 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:30.858328 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a87b9e5-8e58-402e-8ee1-b83130217673 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:30.868047 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1179/2371] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:19:30 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:19:30.872308 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122654d9-20b7-4023-9e03-7e92af30ab17 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:30.882475 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122654d9-20b7-4023-9e03-7e92af30ab17 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:30.888160 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7ab26b-032c-4ef6-ab17-b6bac83d7f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:30.888881 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ab26b-032c-4ef6-ab17-b6bac83d7f48 admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:30.889160 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ab26b-032c-4ef6-ab17-b6bac83d7f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:30.889432 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ab26b-032c-4ef6-ab17-b6bac83d7f48 admin admin] PATH_INFO: `/v3/projects/b6c4982d371844ed8eb589552e65cd85/users/d02424de72204eb0a78d62d688588325/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:30.891307 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7ab26b-032c-4ef6-ab17-b6bac83d7f48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6c4982d371844ed8eb589552e65cd85, user_id=d02424de72204eb0a78d62d688588325, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:30.893975 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7ab26b-032c-4ef6-ab17-b6bac83d7f48 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:30.904691 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1193/2372] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:19:30 2022] PUT /identity/v3/projects/b6c4982d371844ed8eb589552e65cd85/users/d02424de72204eb0a78d62d688588325/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:19:31.718368 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce96411-206b-4956-a532-101be12fd7bb None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:31.719257 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce96411-206b-4956-a532-101be12fd7bb None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:31.719529 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce96411-206b-4956-a532-101be12fd7bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:31.752144 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-cce96411-206b-4956-a532-101be12fd7bb None None] MFA Rules not processed for user `d02424de72204eb0a78d62d688588325`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:19:31.753719 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-cce96411-206b-4956-a532-101be12fd7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:31.840166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1180/2373] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:19:31 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:19:32.953539 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7ab26b-032c-4ef6-ab17-b6bac83d7f48 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:33.030939 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7ab26b-032c-4ef6-ab17-b6bac83d7f48 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:33.035601 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b95a301-9ad2-4b5e-be09-13b78e437d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:33.036246 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b95a301-9ad2-4b5e-be09-13b78e437d30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:33.036527 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b95a301-9ad2-4b5e-be09-13b78e437d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:33.036796 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b95a301-9ad2-4b5e-be09-13b78e437d30 admin admin] PATH_INFO: `/v3/users/0cd24f610d9f4e498e6a7d27185ec02b` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:33.038451 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b95a301-9ad2-4b5e-be09-13b78e437d30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cd24f610d9f4e498e6a7d27185ec02b)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:33.039917 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b95a301-9ad2-4b5e-be09-13b78e437d30 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:33.095275 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-5b95a301-9ad2-4b5e-be09-13b78e437d30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cd24f610d9f4e498e6a7d27185ec02b'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:33.122017 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-5b95a301-9ad2-4b5e-be09-13b78e437d30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cd24f610d9f4e498e6a7d27185ec02b'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:33.122017 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-5b95a301-9ad2-4b5e-be09-13b78e437d30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cd24f610d9f4e498e6a7d27185ec02b'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:33.126318 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-5b95a301-9ad2-4b5e-be09-13b78e437d30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cd24f610d9f4e498e6a7d27185ec02b'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:33.126873 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-5b95a301-9ad2-4b5e-be09-13b78e437d30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cd24f610d9f4e498e6a7d27185ec02b'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:33.133282 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1194/2374] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:19:32 2022] DELETE /identity/v3/users/0cd24f610d9f4e498e6a7d27185ec02b => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:19:33.425028 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce96411-206b-4956-a532-101be12fd7bb None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:33.438072 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce96411-206b-4956-a532-101be12fd7bb None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:33.512784 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154cb90-5f47-42a9-b381-8c73a23c0256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:33.513443 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154cb90-5f47-42a9-b381-8c73a23c0256 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:33.513731 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154cb90-5f47-42a9-b381-8c73a23c0256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:33.514009 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154cb90-5f47-42a9-b381-8c73a23c0256 admin admin] PATH_INFO: `/v3/projects/35752e1b1e3a47e489a160378c591e03` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:33.515212 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e154cb90-5f47-42a9-b381-8c73a23c0256 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35752e1b1e3a47e489a160378c591e03)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:33.517239 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e154cb90-5f47-42a9-b381-8c73a23c0256 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:33.519409 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:19:33.574535 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-e154cb90-5f47-42a9-b381-8c73a23c0256 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35752e1b1e3a47e489a160378c591e03', 'request_id': 'req-e154cb90-5f47-42a9-b381-8c73a23c0256'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:33.583390 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-e154cb90-5f47-42a9-b381-8c73a23c0256 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35752e1b1e3a47e489a160378c591e03', 'request_id': 'req-e154cb90-5f47-42a9-b381-8c73a23c0256'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:33.591087 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-e154cb90-5f47-42a9-b381-8c73a23c0256 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35752e1b1e3a47e489a160378c591e03', 'request_id': 'req-e154cb90-5f47-42a9-b381-8c73a23c0256'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:33.602976 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-e154cb90-5f47-42a9-b381-8c73a23c0256 admin admin] The token cache is being invalidate because project 35752e1b1e3a47e489a160378c591e03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81661) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:19:33.603276 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-e154cb90-5f47-42a9-b381-8c73a23c0256 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:33.603934 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-e154cb90-5f47-42a9-b381-8c73a23c0256 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:33.606772 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1181/2375] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:19:33 2022] DELETE /identity/v3/projects/35752e1b1e3a47e489a160378c591e03 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:19:33.610958 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b95a301-9ad2-4b5e-be09-13b78e437d30 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:33.612502 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-5b95a301-9ad2-4b5e-be09-13b78e437d30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:33.630008 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b95a301-9ad2-4b5e-be09-13b78e437d30 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:33.679514 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:19:33.713765 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ab508f-8a32-4032-ba8d-9b4a1533de74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:33.715141 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ab508f-8a32-4032-ba8d-9b4a1533de74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:33.715423 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ab508f-8a32-4032-ba8d-9b4a1533de74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:33.715856 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ab508f-8a32-4032-ba8d-9b4a1533de74 admin admin] PATH_INFO: `/v3/users/d02424de72204eb0a78d62d688588325` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:33.717085 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ab508f-8a32-4032-ba8d-9b4a1533de74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d02424de72204eb0a78d62d688588325)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:33.719140 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ab508f-8a32-4032-ba8d-9b4a1533de74 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:33.800908 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-b5ab508f-8a32-4032-ba8d-9b4a1533de74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd02424de72204eb0a78d62d688588325'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:33.819261 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-b5ab508f-8a32-4032-ba8d-9b4a1533de74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd02424de72204eb0a78d62d688588325'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:33.827780 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-b5ab508f-8a32-4032-ba8d-9b4a1533de74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd02424de72204eb0a78d62d688588325'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:33.838762 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-b5ab508f-8a32-4032-ba8d-9b4a1533de74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd02424de72204eb0a78d62d688588325'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:33.840208 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-b5ab508f-8a32-4032-ba8d-9b4a1533de74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd02424de72204eb0a78d62d688588325'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:33.849048 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1195/2376] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:19:33 2022] DELETE /identity/v3/users/d02424de72204eb0a78d62d688588325 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:19:34.039839 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154cb90-5f47-42a9-b381-8c73a23c0256 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:34.041368 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-e154cb90-5f47-42a9-b381-8c73a23c0256 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:34.056361 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154cb90-5f47-42a9-b381-8c73a23c0256 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:34.120655 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ba240d-f083-4311-acf5-19b01d9489a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:34.121309 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba240d-f083-4311-acf5-19b01d9489a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:34.121593 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba240d-f083-4311-acf5-19b01d9489a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:34.121868 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba240d-f083-4311-acf5-19b01d9489a3 admin admin] PATH_INFO: `/v3/projects/b6c4982d371844ed8eb589552e65cd85` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:34.123027 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ba240d-f083-4311-acf5-19b01d9489a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6c4982d371844ed8eb589552e65cd85)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:34.125055 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ba240d-f083-4311-acf5-19b01d9489a3 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:34.153087 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ab508f-8a32-4032-ba8d-9b4a1533de74 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:34.154682 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-b5ab508f-8a32-4032-ba8d-9b4a1533de74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:34.170292 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ab508f-8a32-4032-ba8d-9b4a1533de74 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:34.181523 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-e6ba240d-f083-4311-acf5-19b01d9489a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6c4982d371844ed8eb589552e65cd85', 'request_id': 'req-e6ba240d-f083-4311-acf5-19b01d9489a3'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:34.189701 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-e6ba240d-f083-4311-acf5-19b01d9489a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6c4982d371844ed8eb589552e65cd85', 'request_id': 'req-e6ba240d-f083-4311-acf5-19b01d9489a3'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:34.196381 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-e6ba240d-f083-4311-acf5-19b01d9489a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6c4982d371844ed8eb589552e65cd85', 'request_id': 'req-e6ba240d-f083-4311-acf5-19b01d9489a3'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:34.208730 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-e6ba240d-f083-4311-acf5-19b01d9489a3 admin admin] The token cache is being invalidate because project b6c4982d371844ed8eb589552e65cd85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81661) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:19:34.209022 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-e6ba240d-f083-4311-acf5-19b01d9489a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:34.209865 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-e6ba240d-f083-4311-acf5-19b01d9489a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:34.211529 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1182/2377] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:19:34 2022] DELETE /identity/v3/projects/b6c4982d371844ed8eb589552e65cd85 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:19:34.216807 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685cdf6-075f-4e8c-bd58-5bc062a89a79 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:34.217077 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685cdf6-075f-4e8c-bd58-5bc062a89a79 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:34.217369 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685cdf6-075f-4e8c-bd58-5bc062a89a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:34.238990 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-b685cdf6-075f-4e8c-bd58-5bc062a89a79 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:19:34.240252 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-b685cdf6-075f-4e8c-bd58-5bc062a89a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:34.245776 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe2cbdb-63ea-49c4-9e70-71be8a4f80d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:34.246424 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2cbdb-63ea-49c4-9e70-71be8a4f80d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:34.246746 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2cbdb-63ea-49c4-9e70-71be8a4f80d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:34.247018 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2cbdb-63ea-49c4-9e70-71be8a4f80d3 admin admin] PATH_INFO: `/v3/users/a95cb912b58e4e0f9225c6c0511d9105` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:34.248246 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe2cbdb-63ea-49c4-9e70-71be8a4f80d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a95cb912b58e4e0f9225c6c0511d9105)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:34.249447 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1183/2378] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:19:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:19:34.250242 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe2cbdb-63ea-49c4-9e70-71be8a4f80d3 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:34.257293 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f1d4d3-d7c6-468e-a427-5e45836f4620 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:34.257577 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f1d4d3-d7c6-468e-a427-5e45836f4620 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:34.257841 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f1d4d3-d7c6-468e-a427-5e45836f4620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:34.281726 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-37f1d4d3-d7c6-468e-a427-5e45836f4620 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:19:34.282994 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-37f1d4d3-d7c6-468e-a427-5e45836f4620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:34.292993 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1184/2379] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:19:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:19:34.297150 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f1d4d3-d7c6-468e-a427-5e45836f4620 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:34.307543 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f1d4d3-d7c6-468e-a427-5e45836f4620 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:34.313428 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1858541-c967-4e91-a504-07ca485132b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:34.314565 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1858541-c967-4e91-a504-07ca485132b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:34.314717 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-ebe2cbdb-63ea-49c4-9e70-71be8a4f80d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a95cb912b58e4e0f9225c6c0511d9105'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:34.314840 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1858541-c967-4e91-a504-07ca485132b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:34.314840 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1858541-c967-4e91-a504-07ca485132b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:34.315211 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1858541-c967-4e91-a504-07ca485132b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:19:34.315513 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1858541-c967-4e91-a504-07ca485132b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:34.317182 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1858541-c967-4e91-a504-07ca485132b4 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:34.318997 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:19:34.332265 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:19:34.333249 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1185/2380] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:19:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:19:34.337705 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1858541-c967-4e91-a504-07ca485132b4 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:34.337950 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-ebe2cbdb-63ea-49c4-9e70-71be8a4f80d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a95cb912b58e4e0f9225c6c0511d9105'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:34.348058 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-ebe2cbdb-63ea-49c4-9e70-71be8a4f80d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a95cb912b58e4e0f9225c6c0511d9105'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:34.349181 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1858541-c967-4e91-a504-07ca485132b4 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:34.355642 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85dc614-aeab-4fcf-ab1a-da4f1126429d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:34.356398 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85dc614-aeab-4fcf-ab1a-da4f1126429d admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:34.356807 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85dc614-aeab-4fcf-ab1a-da4f1126429d admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:34.357190 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85dc614-aeab-4fcf-ab1a-da4f1126429d admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:34.357811 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85dc614-aeab-4fcf-ab1a-da4f1126429d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:34.358157 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-ebe2cbdb-63ea-49c4-9e70-71be8a4f80d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a95cb912b58e4e0f9225c6c0511d9105'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:34.359077 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-ebe2cbdb-63ea-49c4-9e70-71be8a4f80d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a95cb912b58e4e0f9225c6c0511d9105'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:34.360059 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85dc614-aeab-4fcf-ab1a-da4f1126429d admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:34.368175 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1196/2381] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:19:34 2022] DELETE /identity/v3/users/a95cb912b58e4e0f9225c6c0511d9105 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:19:34.373138 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe1b8b4-ebd1-4dda-94b2-5a8b54174a21 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:34.373494 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe1b8b4-ebd1-4dda-94b2-5a8b54174a21 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:34.374390 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe1b8b4-ebd1-4dda-94b2-5a8b54174a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:34.394370 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-6fe1b8b4-ebd1-4dda-94b2-5a8b54174a21 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:19:34.395788 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-6fe1b8b4-ebd1-4dda-94b2-5a8b54174a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:34.401860 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1186/2382] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:19:34 2022] POST /identity/v3/projects => generated 367 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:19:34.404155 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85dc614-aeab-4fcf-ab1a-da4f1126429d admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:34.407606 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-b85dc614-aeab-4fcf-ab1a-da4f1126429d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:34.486029 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85dc614-aeab-4fcf-ab1a-da4f1126429d admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:34.492157 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1197/2383] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:19:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:19:34.498598 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe1b8b4-ebd1-4dda-94b2-5a8b54174a21 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:34.499917 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-6fe1b8b4-ebd1-4dda-94b2-5a8b54174a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:34.500237 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fada7719-57fc-4d05-91db-5fd53be054d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:34.500879 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada7719-57fc-4d05-91db-5fd53be054d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:34.501163 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada7719-57fc-4d05-91db-5fd53be054d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:34.501438 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada7719-57fc-4d05-91db-5fd53be054d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:34.502131 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fada7719-57fc-4d05-91db-5fd53be054d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:34.504297 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fada7719-57fc-4d05-91db-5fd53be054d3 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:34.527715 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1187/2384] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:19:34 2022] POST /identity/v3/users => generated 284 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 01:19:34.531819 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fada7719-57fc-4d05-91db-5fd53be054d3 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:34.541975 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fada7719-57fc-4d05-91db-5fd53be054d3 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:34.548038 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9033285-b8bf-467d-a499-53ff408a3ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:34.548680 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9033285-b8bf-467d-a499-53ff408a3ae6 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:34.548972 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9033285-b8bf-467d-a499-53ff408a3ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:34.549333 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9033285-b8bf-467d-a499-53ff408a3ae6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:34.550352 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9033285-b8bf-467d-a499-53ff408a3ae6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:34.552762 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9033285-b8bf-467d-a499-53ff408a3ae6 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:34.564252 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1188/2385] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:19:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 01:19:34.568051 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe1b8b4-ebd1-4dda-94b2-5a8b54174a21 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:34.568454 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9033285-b8bf-467d-a499-53ff408a3ae6 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:34.571745 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14faadf8-b72f-4c7f-93dc-9850bcf9291e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:34.572460 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14faadf8-b72f-4c7f-93dc-9850bcf9291e service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:34.572655 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14faadf8-b72f-4c7f-93dc-9850bcf9291e service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:34.572923 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14faadf8-b72f-4c7f-93dc-9850bcf9291e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:34.583058 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9033285-b8bf-467d-a499-53ff408a3ae6 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:34.585429 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14faadf8-b72f-4c7f-93dc-9850bcf9291e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:34.586953 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14faadf8-b72f-4c7f-93dc-9850bcf9291e service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:34.589707 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558dc186-0269-4f0f-8100-478f77e0924b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:34.590602 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558dc186-0269-4f0f-8100-478f77e0924b admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:34.590886 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558dc186-0269-4f0f-8100-478f77e0924b admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:34.591163 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558dc186-0269-4f0f-8100-478f77e0924b admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:34.591817 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558dc186-0269-4f0f-8100-478f77e0924b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:34.593482 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558dc186-0269-4f0f-8100-478f77e0924b admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:34.601890 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1198/2386] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:19:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 01:19:34.606037 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1189/2387] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:19:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 01:19:34.610245 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14faadf8-b72f-4c7f-93dc-9850bcf9291e service neutron] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:34.626376 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14faadf8-b72f-4c7f-93dc-9850bcf9291e service neutron] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:34.632877 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173996bc-5693-476a-974d-068b51dc2ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:34.634122 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173996bc-5693-476a-974d-068b51dc2ec9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:34.634405 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173996bc-5693-476a-974d-068b51dc2ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:34.634677 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173996bc-5693-476a-974d-068b51dc2ec9 admin admin] PATH_INFO: `/v3/projects/9873dfef8df7417983535d8586707af6/users/413b61e52d564bc68a6ccd117b2ac0b5/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:34.652379 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173996bc-5693-476a-974d-068b51dc2ec9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9873dfef8df7417983535d8586707af6, user_id=413b61e52d564bc68a6ccd117b2ac0b5, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:34.655230 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173996bc-5693-476a-974d-068b51dc2ec9 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:34.669322 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1199/2388] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:19:34 2022] PUT /identity/v3/projects/9873dfef8df7417983535d8586707af6/users/413b61e52d564bc68a6ccd117b2ac0b5/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:19:34.681654 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa8cf0-8fd5-4165-9bb7-f13de5e916d5 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:34.681934 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa8cf0-8fd5-4165-9bb7-f13de5e916d5 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:34.682207 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa8cf0-8fd5-4165-9bb7-f13de5e916d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:34.717606 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-5bfa8cf0-8fd5-4165-9bb7-f13de5e916d5 None None] MFA Rules not processed for user `413b61e52d564bc68a6ccd117b2ac0b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:19:34.718912 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-5bfa8cf0-8fd5-4165-9bb7-f13de5e916d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:34.802288 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1190/2389] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:19:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:19:34.807003 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173996bc-5693-476a-974d-068b51dc2ec9 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:34.873902 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:19:34.878188 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173996bc-5693-476a-974d-068b51dc2ec9 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:34.884680 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b459acc-8530-4e63-990d-3ce7d2805cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:34.885365 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b459acc-8530-4e63-990d-3ce7d2805cb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:34.885654 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b459acc-8530-4e63-990d-3ce7d2805cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:34.886533 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b459acc-8530-4e63-990d-3ce7d2805cb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:34.887235 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b459acc-8530-4e63-990d-3ce7d2805cb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:34.889575 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b459acc-8530-4e63-990d-3ce7d2805cb8 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:34.923301 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1200/2390] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:19:34 2022] POST /identity/v3/projects => generated 369 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:19:34.927668 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa8cf0-8fd5-4165-9bb7-f13de5e916d5 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:34.928968 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b459acc-8530-4e63-990d-3ce7d2805cb8 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:34.930917 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-1b459acc-8530-4e63-990d-3ce7d2805cb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:34.951023 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa8cf0-8fd5-4165-9bb7-f13de5e916d5 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:34.989200 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:19:35.028395 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b459acc-8530-4e63-990d-3ce7d2805cb8 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:35.035188 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b94db3a-bf60-4bc2-b084-8a3ff095f6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:35.035763 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce41098a-9551-4af6-bce5-131cb938785f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:35.036075 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b94db3a-bf60-4bc2-b084-8a3ff095f6b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:35.036406 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce41098a-9551-4af6-bce5-131cb938785f admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:35.036679 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b94db3a-bf60-4bc2-b084-8a3ff095f6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:35.036789 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce41098a-9551-4af6-bce5-131cb938785f admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:35.036949 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce41098a-9551-4af6-bce5-131cb938785f admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:35.037476 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b94db3a-bf60-4bc2-b084-8a3ff095f6b7 admin admin] PATH_INFO: `/v3/projects/1fd71ac60f9a4506a9eda527bcce28c8` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:35.037572 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce41098a-9551-4af6-bce5-131cb938785f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:35.039391 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b94db3a-bf60-4bc2-b084-8a3ff095f6b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fd71ac60f9a4506a9eda527bcce28c8)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:35.039534 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce41098a-9551-4af6-bce5-131cb938785f admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:35.041614 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b94db3a-bf60-4bc2-b084-8a3ff095f6b7 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:35.068324 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1191/2391] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:19:34 2022] POST /identity/v3/users => generated 291 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:19:35.069399 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce41098a-9551-4af6-bce5-131cb938785f admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:35.081784 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce41098a-9551-4af6-bce5-131cb938785f admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:35.089289 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f13b1c-01fd-4b8a-af87-4f21fe52fc8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:35.090520 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f13b1c-01fd-4b8a-af87-4f21fe52fc8b admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:35.091171 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f13b1c-01fd-4b8a-af87-4f21fe52fc8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:35.091581 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f13b1c-01fd-4b8a-af87-4f21fe52fc8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:35.092380 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f13b1c-01fd-4b8a-af87-4f21fe52fc8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:35.094225 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f13b1c-01fd-4b8a-af87-4f21fe52fc8b admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:35.106310 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1192/2392] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:19:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:19:35.110416 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f13b1c-01fd-4b8a-af87-4f21fe52fc8b admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:35.121873 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-5b94db3a-bf60-4bc2-b084-8a3ff095f6b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fd71ac60f9a4506a9eda527bcce28c8', 'request_id': 'req-5b94db3a-bf60-4bc2-b084-8a3ff095f6b7'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:35.125746 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f13b1c-01fd-4b8a-af87-4f21fe52fc8b admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:35.128664 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-5b94db3a-bf60-4bc2-b084-8a3ff095f6b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fd71ac60f9a4506a9eda527bcce28c8', 'request_id': 'req-5b94db3a-bf60-4bc2-b084-8a3ff095f6b7'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:35.134664 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-5b94db3a-bf60-4bc2-b084-8a3ff095f6b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fd71ac60f9a4506a9eda527bcce28c8', 'request_id': 'req-5b94db3a-bf60-4bc2-b084-8a3ff095f6b7'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:35.150214 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-5b94db3a-bf60-4bc2-b084-8a3ff095f6b7 admin admin] The token cache is being invalidate because project 1fd71ac60f9a4506a9eda527bcce28c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:19:35.151028 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-5b94db3a-bf60-4bc2-b084-8a3ff095f6b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:35.152217 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-5b94db3a-bf60-4bc2-b084-8a3ff095f6b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:35.154971 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1201/2393] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:19:34 2022] DELETE /identity/v3/projects/1fd71ac60f9a4506a9eda527bcce28c8 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:19:35.159964 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7654f86e-4e2a-4ea3-b386-4690149c4796 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:35.160432 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7654f86e-4e2a-4ea3-b386-4690149c4796 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:35.160716 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7654f86e-4e2a-4ea3-b386-4690149c4796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:35.183977 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-7654f86e-4e2a-4ea3-b386-4690149c4796 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:19:35.185229 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-7654f86e-4e2a-4ea3-b386-4690149c4796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:35.189294 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a372fc-2c5d-4b85-ba6a-7077f1a09587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:35.190074 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a372fc-2c5d-4b85-ba6a-7077f1a09587 admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:35.190355 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a372fc-2c5d-4b85-ba6a-7077f1a09587 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:35.190623 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a372fc-2c5d-4b85-ba6a-7077f1a09587 admin admin] PATH_INFO: `/v3/projects/27b8abc6dd8846b19635c71f8715258d/users/5d1fb52ff6f74e6bad269531e742395c/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:35.193614 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1202/2394] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:19:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:19:35.204557 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354f170-8e34-4b1c-880b-958c6a8ce856 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:35.204841 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354f170-8e34-4b1c-880b-958c6a8ce856 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:35.205114 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354f170-8e34-4b1c-880b-958c6a8ce856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:35.210548 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a372fc-2c5d-4b85-ba6a-7077f1a09587 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27b8abc6dd8846b19635c71f8715258d, user_id=5d1fb52ff6f74e6bad269531e742395c, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:35.213774 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a372fc-2c5d-4b85-ba6a-7077f1a09587 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:35.226850 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-2354f170-8e34-4b1c-880b-958c6a8ce856 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:19:35.228084 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-2354f170-8e34-4b1c-880b-958c6a8ce856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:35.228603 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1193/2395] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:19:35 2022] PUT /identity/v3/projects/27b8abc6dd8846b19635c71f8715258d/users/5d1fb52ff6f74e6bad269531e742395c/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:19:35.232737 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a372fc-2c5d-4b85-ba6a-7077f1a09587 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:35.234899 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-c0a372fc-2c5d-4b85-ba6a-7077f1a09587 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:35.312371 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a372fc-2c5d-4b85-ba6a-7077f1a09587 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:35.316742 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1203/2396] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:19:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:19:35.319065 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2354f170-8e34-4b1c-880b-958c6a8ce856 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:35.321292 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fe0d2a-816b-4084-8386-7e8637414b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:35.322372 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fe0d2a-816b-4084-8386-7e8637414b7c admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:35.328169 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fe0d2a-816b-4084-8386-7e8637414b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:35.328169 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fe0d2a-816b-4084-8386-7e8637414b7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:35.328169 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fe0d2a-816b-4084-8386-7e8637414b7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:35.328169 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fe0d2a-816b-4084-8386-7e8637414b7c admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:35.332495 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2354f170-8e34-4b1c-880b-958c6a8ce856 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:35.338015 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e90352-fa06-4b4b-80c3-f89014f0f67f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:35.338660 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e90352-fa06-4b4b-80c3-f89014f0f67f admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:35.339649 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e90352-fa06-4b4b-80c3-f89014f0f67f admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:35.339933 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e90352-fa06-4b4b-80c3-f89014f0f67f admin admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:35.340617 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e90352-fa06-4b4b-80c3-f89014f0f67f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:19:35.340917 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e90352-fa06-4b4b-80c3-f89014f0f67f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:35.341598 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1194/2397] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:19:35 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 01:19:35.345334 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fe0d2a-816b-4084-8386-7e8637414b7c admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:35.346046 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e90352-fa06-4b4b-80c3-f89014f0f67f admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:35.348081 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:19:35.357821 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fe0d2a-816b-4084-8386-7e8637414b7c admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:35.364346 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3aaffe-7b66-4266-966e-56f91eb7d732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:35.365280 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3aaffe-7b66-4266-966e-56f91eb7d732 admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:35.365561 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3aaffe-7b66-4266-966e-56f91eb7d732 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:35.365839 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3aaffe-7b66-4266-966e-56f91eb7d732 admin admin] PATH_INFO: `/v3/projects/27b8abc6dd8846b19635c71f8715258d/users/5d1fb52ff6f74e6bad269531e742395c/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:35.368298 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1204/2398] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:19:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:19:35.368569 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3aaffe-7b66-4266-966e-56f91eb7d732 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27b8abc6dd8846b19635c71f8715258d, user_id=5d1fb52ff6f74e6bad269531e742395c, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:35.371498 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3aaffe-7b66-4266-966e-56f91eb7d732 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:35.371926 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e90352-fa06-4b4b-80c3-f89014f0f67f admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:35.384246 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e90352-fa06-4b4b-80c3-f89014f0f67f admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:35.388940 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1195/2399] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:19:35 2022] PUT /identity/v3/projects/27b8abc6dd8846b19635c71f8715258d/users/5d1fb52ff6f74e6bad269531e742395c/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:19:35.399532 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c969ed-1210-46f8-822f-3589ddd91e77 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:35.399824 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c969ed-1210-46f8-822f-3589ddd91e77 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:35.400071 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c969ed-1210-46f8-822f-3589ddd91e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:35.429914 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:19:35.433690 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-a3c969ed-1210-46f8-822f-3589ddd91e77 None None] MFA Rules not processed for user `5d1fb52ff6f74e6bad269531e742395c`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:19:35.435083 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-a3c969ed-1210-46f8-822f-3589ddd91e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:35.475770 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb0db6-ff62-43c5-abbe-d18264f03fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:35.477066 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb0db6-ff62-43c5-abbe-d18264f03fee admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:35.477376 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb0db6-ff62-43c5-abbe-d18264f03fee admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:35.477672 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb0db6-ff62-43c5-abbe-d18264f03fee admin admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:35.478329 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bb0db6-ff62-43c5-abbe-d18264f03fee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:35.482185 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bb0db6-ff62-43c5-abbe-d18264f03fee admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:35.515699 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1205/2400] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:19:35 2022] POST /identity/v3/projects => generated 371 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 01:19:35.521053 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb0db6-ff62-43c5-abbe-d18264f03fee admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:35.532610 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1196/2401] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:19:35 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:19:35.542138 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c969ed-1210-46f8-822f-3589ddd91e77 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:35.543702 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-a3c969ed-1210-46f8-822f-3589ddd91e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:35.598606 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb0db6-ff62-43c5-abbe-d18264f03fee admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:35.603831 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed594eb2-f802-498a-86eb-b982a2fc43a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:35.604886 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed594eb2-f802-498a-86eb-b982a2fc43a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:35.605526 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed594eb2-f802-498a-86eb-b982a2fc43a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:35.605957 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed594eb2-f802-498a-86eb-b982a2fc43a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:35.606893 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed594eb2-f802-498a-86eb-b982a2fc43a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:35.609338 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed594eb2-f802-498a-86eb-b982a2fc43a1 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:35.625663 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c969ed-1210-46f8-822f-3589ddd91e77 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:35.649873 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1206/2402] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:19:35 2022] POST /identity/v3/users => generated 286 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:19:35.653023 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed594eb2-f802-498a-86eb-b982a2fc43a1 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:35.673421 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed594eb2-f802-498a-86eb-b982a2fc43a1 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:35.680390 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc65b0-b47d-49ff-a5ee-0507d7cb4d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:35.680998 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc65b0-b47d-49ff-a5ee-0507d7cb4d95 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:35.681273 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc65b0-b47d-49ff-a5ee-0507d7cb4d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:35.681544 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc65b0-b47d-49ff-a5ee-0507d7cb4d95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:35.682206 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bc65b0-b47d-49ff-a5ee-0507d7cb4d95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:35.684618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94e9453-f74f-401e-bd31-3ca2fbbe0f26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:35.684864 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bc65b0-b47d-49ff-a5ee-0507d7cb4d95 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:35.685383 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e9453-f74f-401e-bd31-3ca2fbbe0f26 service cinder] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:35.685661 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e9453-f74f-401e-bd31-3ca2fbbe0f26 service cinder] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:35.685932 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e9453-f74f-401e-bd31-3ca2fbbe0f26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:35.701377 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1207/2403] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:19:35 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:19:35.702475 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc65b0-b47d-49ff-a5ee-0507d7cb4d95 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:35.713667 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc65b0-b47d-49ff-a5ee-0507d7cb4d95 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:35.721054 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae33d80a-e339-48ca-b795-0be291a6dd1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:35.722267 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae33d80a-e339-48ca-b795-0be291a6dd1c admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:35.722745 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae33d80a-e339-48ca-b795-0be291a6dd1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:35.723087 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae33d80a-e339-48ca-b795-0be291a6dd1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:35.724078 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae33d80a-e339-48ca-b795-0be291a6dd1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:35.725705 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae33d80a-e339-48ca-b795-0be291a6dd1c admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:35.737793 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1208/2404] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:19:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:19:35.741859 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae33d80a-e339-48ca-b795-0be291a6dd1c admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:35.751091 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94e9453-f74f-401e-bd31-3ca2fbbe0f26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:35.752651 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae33d80a-e339-48ca-b795-0be291a6dd1c admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:35.752765 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94e9453-f74f-401e-bd31-3ca2fbbe0f26 service cinder] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:35.757233 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d402e-50aa-46d9-ad2b-ca1853763827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:35.757858 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d402e-50aa-46d9-ad2b-ca1853763827 admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:35.758130 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d402e-50aa-46d9-ad2b-ca1853763827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:35.758397 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d402e-50aa-46d9-ad2b-ca1853763827 admin admin] PATH_INFO: `/v3/projects/e32dd1d6dcf54fbb83667953c4cbea29/users/00917dba4a9f4d56be3ec74f90a65154/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:35.763142 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1197/2405] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:19:35 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 01:19:35.775442 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927d402e-50aa-46d9-ad2b-ca1853763827 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e32dd1d6dcf54fbb83667953c4cbea29, user_id=00917dba4a9f4d56be3ec74f90a65154, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:35.778417 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927d402e-50aa-46d9-ad2b-ca1853763827 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:35.793269 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1209/2406] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:19:35 2022] PUT /identity/v3/projects/e32dd1d6dcf54fbb83667953c4cbea29/users/00917dba4a9f4d56be3ec74f90a65154/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:19:35.800074 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21076404-b452-487d-a5af-b807a835bf2a None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:35.800706 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21076404-b452-487d-a5af-b807a835bf2a None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:35.801020 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21076404-b452-487d-a5af-b807a835bf2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:35.811500 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:19:35.823103 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d402e-50aa-46d9-ad2b-ca1853763827 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:35.824863 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-21076404-b452-487d-a5af-b807a835bf2a None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:19:35.825484 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-927d402e-50aa-46d9-ad2b-ca1853763827 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:35.826185 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-21076404-b452-487d-a5af-b807a835bf2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:35.853037 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:19:35.887365 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d402e-50aa-46d9-ad2b-ca1853763827 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:35.889935 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1198/2407] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:19:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:19:35.891001 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bfbd86-a7e7-4d6e-9488-2af7a1284c13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:35.891630 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bfbd86-a7e7-4d6e-9488-2af7a1284c13 service cinder] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:35.891902 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bfbd86-a7e7-4d6e-9488-2af7a1284c13 service cinder] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:35.892199 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bfbd86-a7e7-4d6e-9488-2af7a1284c13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:35.893997 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-09bfbd86-a7e7-4d6e-9488-2af7a1284c13 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:35.899844 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21076404-b452-487d-a5af-b807a835bf2a None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:35.901213 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-21076404-b452-487d-a5af-b807a835bf2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:35.960467 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21076404-b452-487d-a5af-b807a835bf2a None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:35.961853 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bfbd86-a7e7-4d6e-9488-2af7a1284c13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:35.963246 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bfbd86-a7e7-4d6e-9488-2af7a1284c13 service cinder] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:35.964378 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b70ee0-176e-4aea-b15e-cc759ab5f696 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:35.965011 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b70ee0-176e-4aea-b15e-cc759ab5f696 service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:35.965400 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b70ee0-176e-4aea-b15e-cc759ab5f696 service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:35.965675 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b70ee0-176e-4aea-b15e-cc759ab5f696 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:35.972637 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1210/2408] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:19:35 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:19:35.975446 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b70ee0-176e-4aea-b15e-cc759ab5f696 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:35.977016 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b70ee0-176e-4aea-b15e-cc759ab5f696 service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:35.988037 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1199/2409] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:19:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:19:38.154244 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb8a3c-40ad-42d5-83a6-b03bcac434a3 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:38.154742 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb8a3c-40ad-42d5-83a6-b03bcac434a3 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:38.154849 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb8a3c-40ad-42d5-83a6-b03bcac434a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:38.178484 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-37bb8a3c-40ad-42d5-83a6-b03bcac434a3 None None] MFA Rules not processed for user `de75fd7d7721433684cafa2e6d3faaae`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:19:38.179845 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-37bb8a3c-40ad-42d5-83a6-b03bcac434a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:38.252786 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1211/2410] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Sep 8 01:19:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:19:38.269348 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b70ee0-176e-4aea-b15e-cc759ab5f696 service neutron] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:38.270935 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-37b70ee0-176e-4aea-b15e-cc759ab5f696 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:38.355434 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b70ee0-176e-4aea-b15e-cc759ab5f696 service neutron] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:38.361458 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f40b8c-e072-439e-b1a2-c86246ed8a79 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8add26872fe6479aa9d7de155a17195a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:38.362167 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f40b8c-e072-439e-b1a2-c86246ed8a79 service swift] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:38.362518 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f40b8c-e072-439e-b1a2-c86246ed8a79 service swift] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:38.363232 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f40b8c-e072-439e-b1a2-c86246ed8a79 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:38.377339 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f40b8c-e072-439e-b1a2-c86246ed8a79 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:38.379069 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f40b8c-e072-439e-b1a2-c86246ed8a79 service swift] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:38.391720 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1200/2411] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Sep 8 01:19:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 01:19:40.007229 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bb8a3c-40ad-42d5-83a6-b03bcac434a3 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:40.041844 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bb8a3c-40ad-42d5-83a6-b03bcac434a3 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:40.051152 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b51e25f-66a8-4a7e-b2a1-f1b73aa6ab4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:40.051853 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51e25f-66a8-4a7e-b2a1-f1b73aa6ab4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:40.052160 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51e25f-66a8-4a7e-b2a1-f1b73aa6ab4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:40.052489 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51e25f-66a8-4a7e-b2a1-f1b73aa6ab4f admin admin] PATH_INFO: `/v3/users/413b61e52d564bc68a6ccd117b2ac0b5` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:40.053816 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b51e25f-66a8-4a7e-b2a1-f1b73aa6ab4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=413b61e52d564bc68a6ccd117b2ac0b5)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:40.055769 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b51e25f-66a8-4a7e-b2a1-f1b73aa6ab4f admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:40.156798 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-9b51e25f-66a8-4a7e-b2a1-f1b73aa6ab4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '413b61e52d564bc68a6ccd117b2ac0b5'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:40.186905 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-9b51e25f-66a8-4a7e-b2a1-f1b73aa6ab4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '413b61e52d564bc68a6ccd117b2ac0b5'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:40.201007 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-9b51e25f-66a8-4a7e-b2a1-f1b73aa6ab4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '413b61e52d564bc68a6ccd117b2ac0b5'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:40.205316 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:19:40.216049 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-9b51e25f-66a8-4a7e-b2a1-f1b73aa6ab4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '413b61e52d564bc68a6ccd117b2ac0b5'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:40.216749 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-9b51e25f-66a8-4a7e-b2a1-f1b73aa6ab4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '413b61e52d564bc68a6ccd117b2ac0b5'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:40.231127 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1212/2412] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:19:40 2022] DELETE /identity/v3/users/413b61e52d564bc68a6ccd117b2ac0b5 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:19:40.236667 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff18df-2315-4644-9f51-5c99f328fd08 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:40.237578 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff18df-2315-4644-9f51-5c99f328fd08 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:40.238611 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff18df-2315-4644-9f51-5c99f328fd08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:40.261900 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-30ff18df-2315-4644-9f51-5c99f328fd08 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:19:40.263154 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-30ff18df-2315-4644-9f51-5c99f328fd08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:40.334916 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:19:40.417127 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1201/2413] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:19:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:19:40.417900 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b51e25f-66a8-4a7e-b2a1-f1b73aa6ab4f admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:40.420228 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-9b51e25f-66a8-4a7e-b2a1-f1b73aa6ab4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:40.540614 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b51e25f-66a8-4a7e-b2a1-f1b73aa6ab4f admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:40.544652 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9936e042-046a-4281-a0fa-cac1e0f04374 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:40.545271 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936e042-046a-4281-a0fa-cac1e0f04374 service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:40.545548 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936e042-046a-4281-a0fa-cac1e0f04374 service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:40.545824 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936e042-046a-4281-a0fa-cac1e0f04374 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:40.559351 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9936e042-046a-4281-a0fa-cac1e0f04374 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:40.561544 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9936e042-046a-4281-a0fa-cac1e0f04374 service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:40.577595 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1213/2414] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:19:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:19:41.188260 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ff18df-2315-4644-9f51-5c99f328fd08 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:41.189913 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-30ff18df-2315-4644-9f51-5c99f328fd08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:41.209875 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ff18df-2315-4644-9f51-5c99f328fd08 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:41.217561 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19c351b-4466-45b9-b86b-3d84bf74adc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:41.218538 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c351b-4466-45b9-b86b-3d84bf74adc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:41.219008 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c351b-4466-45b9-b86b-3d84bf74adc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:41.219354 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c351b-4466-45b9-b86b-3d84bf74adc6 admin admin] PATH_INFO: `/v3/projects/9873dfef8df7417983535d8586707af6` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:41.220708 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19c351b-4466-45b9-b86b-3d84bf74adc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9873dfef8df7417983535d8586707af6)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:41.223194 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19c351b-4466-45b9-b86b-3d84bf74adc6 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:41.317323 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-c19c351b-4466-45b9-b86b-3d84bf74adc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9873dfef8df7417983535d8586707af6', 'request_id': 'req-c19c351b-4466-45b9-b86b-3d84bf74adc6'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:41.329131 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-c19c351b-4466-45b9-b86b-3d84bf74adc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9873dfef8df7417983535d8586707af6', 'request_id': 'req-c19c351b-4466-45b9-b86b-3d84bf74adc6'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:41.343606 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-c19c351b-4466-45b9-b86b-3d84bf74adc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9873dfef8df7417983535d8586707af6', 'request_id': 'req-c19c351b-4466-45b9-b86b-3d84bf74adc6'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:41.359394 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-c19c351b-4466-45b9-b86b-3d84bf74adc6 admin admin] The token cache is being invalidate because project 9873dfef8df7417983535d8586707af6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81661) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:19:41.359958 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-c19c351b-4466-45b9-b86b-3d84bf74adc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:41.360854 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-c19c351b-4466-45b9-b86b-3d84bf74adc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:41.365936 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1202/2415] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:19:41 2022] DELETE /identity/v3/projects/9873dfef8df7417983535d8586707af6 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:19:41.367134 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9936e042-046a-4281-a0fa-cac1e0f04374 service neutron] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:41.406391 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9936e042-046a-4281-a0fa-cac1e0f04374 service neutron] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:41.543091 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be497e87-e393-44e5-8047-b65bffd99fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:41.543866 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be497e87-e393-44e5-8047-b65bffd99fd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:41.544278 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be497e87-e393-44e5-8047-b65bffd99fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:41.544654 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be497e87-e393-44e5-8047-b65bffd99fd7 admin admin] PATH_INFO: `/v3/users/5d1fb52ff6f74e6bad269531e742395c` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:41.545901 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be497e87-e393-44e5-8047-b65bffd99fd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d1fb52ff6f74e6bad269531e742395c)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:41.549376 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be497e87-e393-44e5-8047-b65bffd99fd7 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:41.643792 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-be497e87-e393-44e5-8047-b65bffd99fd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d1fb52ff6f74e6bad269531e742395c'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:41.651855 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177163b-3fd9-40bf-8ce1-b87c8b6763a9 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:41.652318 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177163b-3fd9-40bf-8ce1-b87c8b6763a9 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:41.652833 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177163b-3fd9-40bf-8ce1-b87c8b6763a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:41.687237 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-be497e87-e393-44e5-8047-b65bffd99fd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d1fb52ff6f74e6bad269531e742395c'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:41.706123 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-be497e87-e393-44e5-8047-b65bffd99fd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d1fb52ff6f74e6bad269531e742395c'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:41.717501 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-c177163b-3fd9-40bf-8ce1-b87c8b6763a9 None None] MFA Rules not processed for user `00917dba4a9f4d56be3ec74f90a65154`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:19:41.719100 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-c177163b-3fd9-40bf-8ce1-b87c8b6763a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:41.722460 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-be497e87-e393-44e5-8047-b65bffd99fd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d1fb52ff6f74e6bad269531e742395c'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:41.723483 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-be497e87-e393-44e5-8047-b65bffd99fd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d1fb52ff6f74e6bad269531e742395c'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:41.738984 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1214/2416] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:19:41 2022] DELETE /identity/v3/users/5d1fb52ff6f74e6bad269531e742395c => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:19:41.904022 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1203/2417] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:19:41 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:19:41.911689 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be497e87-e393-44e5-8047-b65bffd99fd7 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:41.913969 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-be497e87-e393-44e5-8047-b65bffd99fd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:41.959025 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:19:42.084582 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be497e87-e393-44e5-8047-b65bffd99fd7 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:42.089938 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bcb47f-44ac-43ce-89b2-2e73103dc3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:42.090785 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bcb47f-44ac-43ce-89b2-2e73103dc3f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:42.091146 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bcb47f-44ac-43ce-89b2-2e73103dc3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:42.091494 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bcb47f-44ac-43ce-89b2-2e73103dc3f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:42.092248 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bcb47f-44ac-43ce-89b2-2e73103dc3f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:42.108917 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bcb47f-44ac-43ce-89b2-2e73103dc3f7 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:42.151624 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1215/2418] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:19:41 2022] POST /identity/v3/projects => generated 373 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:19:42.157607 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c177163b-3fd9-40bf-8ce1-b87c8b6763a9 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:42.209422 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c177163b-3fd9-40bf-8ce1-b87c8b6763a9 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:42.308115 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bcb47f-44ac-43ce-89b2-2e73103dc3f7 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:42.309903 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-83bcb47f-44ac-43ce-89b2-2e73103dc3f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:42.353187 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:19:42.386552 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2725904f-7486-4e64-a850-a7166af0780b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:42.387329 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2725904f-7486-4e64-a850-a7166af0780b admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:42.387726 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2725904f-7486-4e64-a850-a7166af0780b admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:42.388538 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2725904f-7486-4e64-a850-a7166af0780b admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:42.389688 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2725904f-7486-4e64-a850-a7166af0780b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:42.393633 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2725904f-7486-4e64-a850-a7166af0780b admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:42.446647 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1204/2419] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:19:42 2022] POST /identity/v3/users => generated 293 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:19:42.457279 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bcb47f-44ac-43ce-89b2-2e73103dc3f7 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:42.462744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2725904f-7486-4e64-a850-a7166af0780b admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:42.469129 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ce7edf-ff83-42dd-9481-3c974f3373f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:42.469801 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce7edf-ff83-42dd-9481-3c974f3373f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:42.470087 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce7edf-ff83-42dd-9481-3c974f3373f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:42.470365 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce7edf-ff83-42dd-9481-3c974f3373f2 admin admin] PATH_INFO: `/v3/projects/27b8abc6dd8846b19635c71f8715258d` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:42.474882 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ce7edf-ff83-42dd-9481-3c974f3373f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27b8abc6dd8846b19635c71f8715258d)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:42.482136 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ce7edf-ff83-42dd-9481-3c974f3373f2 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:42.484733 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2725904f-7486-4e64-a850-a7166af0780b admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:42.491070 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bad348e-e343-4b79-a661-8cf240b09b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:42.491702 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bad348e-e343-4b79-a661-8cf240b09b8c admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:42.491985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bad348e-e343-4b79-a661-8cf240b09b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:42.492358 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bad348e-e343-4b79-a661-8cf240b09b8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:42.493020 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bad348e-e343-4b79-a661-8cf240b09b8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:42.494882 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bad348e-e343-4b79-a661-8cf240b09b8c admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:42.513315 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1205/2420] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:19:42 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:19:42.514767 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bad348e-e343-4b79-a661-8cf240b09b8c admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:42.528369 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bad348e-e343-4b79-a661-8cf240b09b8c admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:42.537531 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a280510a-ae29-4c7c-82c0-aefc21d1f871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:42.538753 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a280510a-ae29-4c7c-82c0-aefc21d1f871 admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:42.539147 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a280510a-ae29-4c7c-82c0-aefc21d1f871 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:42.539524 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a280510a-ae29-4c7c-82c0-aefc21d1f871 admin admin] PATH_INFO: `/v3/projects/4037ed70420a475cb2a0a50d75d15daf/users/fca710369bd546599a21503e3fb505dc/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:42.561006 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a280510a-ae29-4c7c-82c0-aefc21d1f871 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4037ed70420a475cb2a0a50d75d15daf, user_id=fca710369bd546599a21503e3fb505dc, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:42.564656 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a280510a-ae29-4c7c-82c0-aefc21d1f871 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:42.595317 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-56ce7edf-ff83-42dd-9481-3c974f3373f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27b8abc6dd8846b19635c71f8715258d', 'request_id': 'req-56ce7edf-ff83-42dd-9481-3c974f3373f2'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:42.596584 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1206/2421] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:19:42 2022] PUT /identity/v3/projects/4037ed70420a475cb2a0a50d75d15daf/users/fca710369bd546599a21503e3fb505dc/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:19:42.598119 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a280510a-ae29-4c7c-82c0-aefc21d1f871 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:42.614801 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-56ce7edf-ff83-42dd-9481-3c974f3373f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27b8abc6dd8846b19635c71f8715258d', 'request_id': 'req-56ce7edf-ff83-42dd-9481-3c974f3373f2'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:42.628151 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-56ce7edf-ff83-42dd-9481-3c974f3373f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27b8abc6dd8846b19635c71f8715258d', 'request_id': 'req-56ce7edf-ff83-42dd-9481-3c974f3373f2'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:42.651178 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-56ce7edf-ff83-42dd-9481-3c974f3373f2 admin admin] The token cache is being invalidate because project 27b8abc6dd8846b19635c71f8715258d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:19:42.653298 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-56ce7edf-ff83-42dd-9481-3c974f3373f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:42.654755 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-56ce7edf-ff83-42dd-9481-3c974f3373f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:42.660905 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1216/2422] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:19:42 2022] DELETE /identity/v3/projects/27b8abc6dd8846b19635c71f8715258d => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:19:42.666307 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3fb68-c1e3-4492-90da-491fb7211135 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:42.668361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3fb68-c1e3-4492-90da-491fb7211135 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:42.668828 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3fb68-c1e3-4492-90da-491fb7211135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:42.718621 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a280510a-ae29-4c7c-82c0-aefc21d1f871 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:42.720814 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-34e3fb68-c1e3-4492-90da-491fb7211135 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:19:42.724737 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-34e3fb68-c1e3-4492-90da-491fb7211135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:42.729995 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428a1311-eb32-4b68-894f-1100659d1da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:42.730838 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428a1311-eb32-4b68-894f-1100659d1da4 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:42.731072 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428a1311-eb32-4b68-894f-1100659d1da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:42.731433 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428a1311-eb32-4b68-894f-1100659d1da4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:42.732342 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428a1311-eb32-4b68-894f-1100659d1da4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:42.734312 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428a1311-eb32-4b68-894f-1100659d1da4 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:42.752684 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1217/2423] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:19:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:19:42.757501 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1207/2424] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:19:42 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:19:42.757668 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da6175-6306-4d3a-bc69-3b4c85414ec2 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:42.757668 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da6175-6306-4d3a-bc69-3b4c85414ec2 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:42.757668 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da6175-6306-4d3a-bc69-3b4c85414ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:42.759344 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428a1311-eb32-4b68-894f-1100659d1da4 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:42.763117 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-428a1311-eb32-4b68-894f-1100659d1da4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:42.782715 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-91da6175-6306-4d3a-bc69-3b4c85414ec2 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:19:42.784146 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-91da6175-6306-4d3a-bc69-3b4c85414ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:42.784604 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428a1311-eb32-4b68-894f-1100659d1da4 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:42.792665 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3304fe16-ee7b-4535-8de0-b172d0e987df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:42.793592 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3304fe16-ee7b-4535-8de0-b172d0e987df admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:42.793895 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3304fe16-ee7b-4535-8de0-b172d0e987df admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:42.797162 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3304fe16-ee7b-4535-8de0-b172d0e987df admin admin] PATH_INFO: `/v3/projects/4037ed70420a475cb2a0a50d75d15daf/users/fca710369bd546599a21503e3fb505dc/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:42.797296 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1218/2425] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:19:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:19:42.797393 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3304fe16-ee7b-4535-8de0-b172d0e987df admin admin] RBAC: Authorizing `identity:create_grant(project_id=4037ed70420a475cb2a0a50d75d15daf, user_id=fca710369bd546599a21503e3fb505dc, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:42.798878 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da6175-6306-4d3a-bc69-3b4c85414ec2 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:42.799947 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3304fe16-ee7b-4535-8de0-b172d0e987df admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:42.815024 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da6175-6306-4d3a-bc69-3b4c85414ec2 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:42.815943 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1208/2426] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:19:42 2022] PUT /identity/v3/projects/4037ed70420a475cb2a0a50d75d15daf/users/fca710369bd546599a21503e3fb505dc/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:19:42.842247 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:19:42.926552 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3ffe20-341d-4f1b-8798-20bef537ee2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:42.927945 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ffe20-341d-4f1b-8798-20bef537ee2e admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:42.928816 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ffe20-341d-4f1b-8798-20bef537ee2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:42.929584 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ffe20-341d-4f1b-8798-20bef537ee2e admin admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:42.930786 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3ffe20-341d-4f1b-8798-20bef537ee2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:19:42.931343 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3ffe20-341d-4f1b-8798-20bef537ee2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:42.937977 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3ffe20-341d-4f1b-8798-20bef537ee2e admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:42.964313 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1219/2427] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:19:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:19:42.964781 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3304fe16-ee7b-4535-8de0-b172d0e987df admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:42.981664 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3304fe16-ee7b-4535-8de0-b172d0e987df admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:43.037065 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441ad212-fc00-4537-8e65-b2c769413d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:43.037831 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441ad212-fc00-4537-8e65-b2c769413d44 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:43.038440 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441ad212-fc00-4537-8e65-b2c769413d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:43.039438 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441ad212-fc00-4537-8e65-b2c769413d44 admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:43.040720 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441ad212-fc00-4537-8e65-b2c769413d44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:43.043856 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441ad212-fc00-4537-8e65-b2c769413d44 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:43.118373 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1209/2428] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:19:42 2022] POST /identity/v3/projects => generated 377 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:19:43.122718 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3ffe20-341d-4f1b-8798-20bef537ee2e admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:43.133849 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3ffe20-341d-4f1b-8798-20bef537ee2e admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:43.241372 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc889ed7-e26f-49f1-82fa-e4d4702c5ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:43.242063 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc889ed7-e26f-49f1-82fa-e4d4702c5ad3 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:43.242405 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc889ed7-e26f-49f1-82fa-e4d4702c5ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:43.242799 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc889ed7-e26f-49f1-82fa-e4d4702c5ad3 admin admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:43.243501 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc889ed7-e26f-49f1-82fa-e4d4702c5ad3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:43.246734 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc889ed7-e26f-49f1-82fa-e4d4702c5ad3 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:43.289872 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441ad212-fc00-4537-8e65-b2c769413d44 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:43.297453 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1220/2429] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:19:43 2022] POST /identity/v3/users => generated 289 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:19:43.302445 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441ad212-fc00-4537-8e65-b2c769413d44 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:43.309528 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b733377-a7b5-4b40-a33b-1dd5ddca9811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:43.310326 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b733377-a7b5-4b40-a33b-1dd5ddca9811 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:43.310764 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b733377-a7b5-4b40-a33b-1dd5ddca9811 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:43.311201 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b733377-a7b5-4b40-a33b-1dd5ddca9811 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:43.312014 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b733377-a7b5-4b40-a33b-1dd5ddca9811 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:43.314236 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b733377-a7b5-4b40-a33b-1dd5ddca9811 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:43.336208 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1210/2430] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:19:43 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:19:43.338382 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc889ed7-e26f-49f1-82fa-e4d4702c5ad3 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:43.352278 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc889ed7-e26f-49f1-82fa-e4d4702c5ad3 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:43.365814 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a91741-7de6-4ef5-9855-a34f18db4464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:43.366622 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a91741-7de6-4ef5-9855-a34f18db4464 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:43.367066 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a91741-7de6-4ef5-9855-a34f18db4464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:43.367493 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a91741-7de6-4ef5-9855-a34f18db4464 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:43.369253 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a91741-7de6-4ef5-9855-a34f18db4464 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:43.372023 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a91741-7de6-4ef5-9855-a34f18db4464 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:43.413166 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1221/2431] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:19:43 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:19:43.414319 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b733377-a7b5-4b40-a33b-1dd5ddca9811 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:43.427852 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b733377-a7b5-4b40-a33b-1dd5ddca9811 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:43.442314 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65847d-cfaa-455e-8344-d5d7c9861639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:43.443191 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65847d-cfaa-455e-8344-d5d7c9861639 admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:43.443615 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65847d-cfaa-455e-8344-d5d7c9861639 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:43.444051 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65847d-cfaa-455e-8344-d5d7c9861639 admin admin] PATH_INFO: `/v3/projects/6de95c9f7d1948b895fb2830c4d5af4b/users/9c8f0dc722c746b081f7d0b83347acc9/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:43.472290 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e65847d-cfaa-455e-8344-d5d7c9861639 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6de95c9f7d1948b895fb2830c4d5af4b, user_id=9c8f0dc722c746b081f7d0b83347acc9, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:43.475400 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e65847d-cfaa-455e-8344-d5d7c9861639 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:43.508187 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1211/2432] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:19:43 2022] PUT /identity/v3/projects/6de95c9f7d1948b895fb2830c4d5af4b/users/9c8f0dc722c746b081f7d0b83347acc9/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:19:43.522106 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb1c54-7224-4c34-8255-85c3cc64f876 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:43.522664 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb1c54-7224-4c34-8255-85c3cc64f876 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:43.522664 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb1c54-7224-4c34-8255-85c3cc64f876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:43.565168 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-aefb1c54-7224-4c34-8255-85c3cc64f876 None None] MFA Rules not processed for user `9c8f0dc722c746b081f7d0b83347acc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:19:43.566768 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-aefb1c54-7224-4c34-8255-85c3cc64f876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:43.786187 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1222/2433] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:19:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:19:43.793109 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65847d-cfaa-455e-8344-d5d7c9861639 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:43.857250 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:19:43.906688 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65847d-cfaa-455e-8344-d5d7c9861639 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:43.920751 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a1592-bbe9-4dc5-925b-a7de5ebce420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:43.921169 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a1592-bbe9-4dc5-925b-a7de5ebce420 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:43.921665 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a1592-bbe9-4dc5-925b-a7de5ebce420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:43.921985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a1592-bbe9-4dc5-925b-a7de5ebce420 admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:43.924474 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380a1592-bbe9-4dc5-925b-a7de5ebce420 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:43.926925 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380a1592-bbe9-4dc5-925b-a7de5ebce420 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:44.014644 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1212/2434] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:19:43 2022] POST /identity/v3/projects => generated 377 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:19:44.018574 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb1c54-7224-4c34-8255-85c3cc64f876 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:44.032348 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb1c54-7224-4c34-8255-85c3cc64f876 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:44.067062 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:19:44.139335 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a17cb4-b180-4bd7-a204-fd89edb5a5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:44.140164 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a17cb4-b180-4bd7-a204-fd89edb5a5ad admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:44.140660 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a17cb4-b180-4bd7-a204-fd89edb5a5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:44.141450 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a17cb4-b180-4bd7-a204-fd89edb5a5ad admin admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:44.142419 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a17cb4-b180-4bd7-a204-fd89edb5a5ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:44.144770 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a17cb4-b180-4bd7-a204-fd89edb5a5ad admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:44.173638 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1223/2435] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:19:44 2022] POST /identity/v3/users => generated 289 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 01:19:44.177645 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a1592-bbe9-4dc5-925b-a7de5ebce420 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:44.195909 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a1592-bbe9-4dc5-925b-a7de5ebce420 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:44.210074 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8bfa04-265b-49a9-87b3-aea35a94c8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:44.211326 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8bfa04-265b-49a9-87b3-aea35a94c8be admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:44.211978 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8bfa04-265b-49a9-87b3-aea35a94c8be admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:44.212484 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8bfa04-265b-49a9-87b3-aea35a94c8be admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:44.213398 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8bfa04-265b-49a9-87b3-aea35a94c8be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:44.216006 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8bfa04-265b-49a9-87b3-aea35a94c8be admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:44.230204 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1213/2436] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:19:44 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:19:44.234145 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a17cb4-b180-4bd7-a204-fd89edb5a5ad admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:44.248856 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a17cb4-b180-4bd7-a204-fd89edb5a5ad admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:44.262991 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600337eb-a179-4817-a40b-fb8c22bdb44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:44.264342 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600337eb-a179-4817-a40b-fb8c22bdb44d admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:44.264625 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600337eb-a179-4817-a40b-fb8c22bdb44d admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:44.264905 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600337eb-a179-4817-a40b-fb8c22bdb44d admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:44.265567 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600337eb-a179-4817-a40b-fb8c22bdb44d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:44.267196 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600337eb-a179-4817-a40b-fb8c22bdb44d admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:44.300592 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1224/2437] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:19:44 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:19:44.304969 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8bfa04-265b-49a9-87b3-aea35a94c8be admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:44.318760 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8bfa04-265b-49a9-87b3-aea35a94c8be admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:44.328492 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316682e8-e55a-4682-bb81-731c77ce59d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:44.329185 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316682e8-e55a-4682-bb81-731c77ce59d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:44.329467 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316682e8-e55a-4682-bb81-731c77ce59d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:44.329918 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316682e8-e55a-4682-bb81-731c77ce59d6 admin admin] PATH_INFO: `/v3/projects/3c6458a01d2f4cb093ef9c2c9ddd9e1c/users/eaad5576abe6475cb110dcbb03d1e72c/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:44.349318 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316682e8-e55a-4682-bb81-731c77ce59d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c6458a01d2f4cb093ef9c2c9ddd9e1c, user_id=eaad5576abe6475cb110dcbb03d1e72c, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:44.352380 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316682e8-e55a-4682-bb81-731c77ce59d6 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:44.371228 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1214/2438] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:19:44 2022] PUT /identity/v3/projects/3c6458a01d2f4cb093ef9c2c9ddd9e1c/users/eaad5576abe6475cb110dcbb03d1e72c/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:19:44.382505 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f737f06-6bad-4306-96a4-68f8b9473cd1 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:44.382808 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f737f06-6bad-4306-96a4-68f8b9473cd1 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:44.383100 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f737f06-6bad-4306-96a4-68f8b9473cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:44.421397 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-1f737f06-6bad-4306-96a4-68f8b9473cd1 None None] MFA Rules not processed for user `eaad5576abe6475cb110dcbb03d1e72c`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:19:44.422641 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-1f737f06-6bad-4306-96a4-68f8b9473cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:44.580831 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1225/2439] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:19:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:19:44.585527 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316682e8-e55a-4682-bb81-731c77ce59d6 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:44.756601 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316682e8-e55a-4682-bb81-731c77ce59d6 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:44.770901 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac92ee71-e11a-4693-8842-9451e28a3d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:44.771539 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92ee71-e11a-4693-8842-9451e28a3d45 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:44.771817 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92ee71-e11a-4693-8842-9451e28a3d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:44.772084 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92ee71-e11a-4693-8842-9451e28a3d45 admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:44.790646 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac92ee71-e11a-4693-8842-9451e28a3d45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:44.794246 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac92ee71-e11a-4693-8842-9451e28a3d45 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:44.881437 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1215/2440] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:19:44 2022] POST /identity/v3/projects => generated 377 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 01:19:44.882155 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f737f06-6bad-4306-96a4-68f8b9473cd1 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:44.902409 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f737f06-6bad-4306-96a4-68f8b9473cd1 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:45.104772 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b88333-2f25-4467-8bdd-fd436d8a3a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:45.105405 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b88333-2f25-4467-8bdd-fd436d8a3a4d admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:45.105683 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b88333-2f25-4467-8bdd-fd436d8a3a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:45.106724 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b88333-2f25-4467-8bdd-fd436d8a3a4d admin admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:45.107235 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b88333-2f25-4467-8bdd-fd436d8a3a4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:45.109943 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b88333-2f25-4467-8bdd-fd436d8a3a4d admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:45.159894 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1226/2441] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:19:44 2022] POST /identity/v3/users => generated 295 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 01:19:45.164585 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac92ee71-e11a-4693-8842-9451e28a3d45 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:45.192419 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac92ee71-e11a-4693-8842-9451e28a3d45 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:45.217727 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1963ff-3c85-4271-8f5e-ac637b63b8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:45.218663 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1963ff-3c85-4271-8f5e-ac637b63b8e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:45.218927 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1963ff-3c85-4271-8f5e-ac637b63b8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:45.219299 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1963ff-3c85-4271-8f5e-ac637b63b8e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:45.220830 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1963ff-3c85-4271-8f5e-ac637b63b8e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:45.230360 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1963ff-3c85-4271-8f5e-ac637b63b8e8 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:45.259392 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1216/2442] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:19:45 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:19:45.260258 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b88333-2f25-4467-8bdd-fd436d8a3a4d admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:45.273495 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b88333-2f25-4467-8bdd-fd436d8a3a4d admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:45.292722 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6574b0-9a0a-4b77-b3fd-a9b1002e5625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:45.293417 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6574b0-9a0a-4b77-b3fd-a9b1002e5625 admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:45.293699 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6574b0-9a0a-4b77-b3fd-a9b1002e5625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:45.293974 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6574b0-9a0a-4b77-b3fd-a9b1002e5625 admin admin] PATH_INFO: `/v3/projects/39aec913db2f48f68f4121749cffe2f8/users/2f6acecfc8724b969ae5862e3519a80a/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:45.349396 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6574b0-9a0a-4b77-b3fd-a9b1002e5625 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39aec913db2f48f68f4121749cffe2f8, user_id=2f6acecfc8724b969ae5862e3519a80a, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:45.352168 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6574b0-9a0a-4b77-b3fd-a9b1002e5625 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:45.355610 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:19:45.410249 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1227/2443] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:19:45 2022] PUT /identity/v3/projects/39aec913db2f48f68f4121749cffe2f8/users/2f6acecfc8724b969ae5862e3519a80a/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:19:45.414829 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1963ff-3c85-4271-8f5e-ac637b63b8e8 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:45.427595 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1963ff-3c85-4271-8f5e-ac637b63b8e8 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:45.589188 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:19:45.616918 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe87b37-978a-4586-8a65-76e7968882a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:45.617562 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe87b37-978a-4586-8a65-76e7968882a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:45.617827 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe87b37-978a-4586-8a65-76e7968882a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:45.618414 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe87b37-978a-4586-8a65-76e7968882a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:45.619208 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe87b37-978a-4586-8a65-76e7968882a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:45.623949 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe87b37-978a-4586-8a65-76e7968882a8 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:45.655980 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1217/2444] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:19:45 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:19:45.657754 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6574b0-9a0a-4b77-b3fd-a9b1002e5625 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:45.675015 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6574b0-9a0a-4b77-b3fd-a9b1002e5625 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:45.698110 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e19f7d5-38b0-400f-9a9f-e84d1e084b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:45.699237 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e19f7d5-38b0-400f-9a9f-e84d1e084b5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:45.700547 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e19f7d5-38b0-400f-9a9f-e84d1e084b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:45.700900 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e19f7d5-38b0-400f-9a9f-e84d1e084b5f admin admin] PATH_INFO: `/v3/projects/39aec913db2f48f68f4121749cffe2f8/users/2f6acecfc8724b969ae5862e3519a80a/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:45.708126 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e19f7d5-38b0-400f-9a9f-e84d1e084b5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=39aec913db2f48f68f4121749cffe2f8, user_id=2f6acecfc8724b969ae5862e3519a80a, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:45.716303 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e19f7d5-38b0-400f-9a9f-e84d1e084b5f admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:45.772467 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1228/2445] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:19:45 2022] PUT /identity/v3/projects/39aec913db2f48f68f4121749cffe2f8/users/2f6acecfc8724b969ae5862e3519a80a/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:19:45.786605 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f157c88-c678-4eed-b665-c3bb1749d105 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:45.787126 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f157c88-c678-4eed-b665-c3bb1749d105 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:45.787585 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f157c88-c678-4eed-b665-c3bb1749d105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:45.859045 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-7f157c88-c678-4eed-b665-c3bb1749d105 None None] MFA Rules not processed for user `2f6acecfc8724b969ae5862e3519a80a`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:19:45.861569 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-7f157c88-c678-4eed-b665-c3bb1749d105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:45.994008 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1218/2446] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:19:45 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:19:45.995047 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e19f7d5-38b0-400f-9a9f-e84d1e084b5f admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:45.997102 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-9e19f7d5-38b0-400f-9a9f-e84d1e084b5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:46.099712 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e19f7d5-38b0-400f-9a9f-e84d1e084b5f admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:46.116119 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094b69da-1da5-47de-9947-641c1537bdfe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:46.116846 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b69da-1da5-47de-9947-641c1537bdfe service cinder] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:46.117131 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b69da-1da5-47de-9947-641c1537bdfe service cinder] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:46.117444 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b69da-1da5-47de-9947-641c1537bdfe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:46.230582 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094b69da-1da5-47de-9947-641c1537bdfe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:46.236354 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094b69da-1da5-47de-9947-641c1537bdfe service cinder] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:46.262405 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1229/2447] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:19:45 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:19:48.447424 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f157c88-c678-4eed-b665-c3bb1749d105 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:48.452734 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-7f157c88-c678-4eed-b665-c3bb1749d105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:48.476249 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f157c88-c678-4eed-b665-c3bb1749d105 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:48.481428 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca722d6-2996-44a5-bb8a-f849beeaca00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:48.482203 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca722d6-2996-44a5-bb8a-f849beeaca00 service cinder] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:48.482488 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca722d6-2996-44a5-bb8a-f849beeaca00 service cinder] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:48.482759 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca722d6-2996-44a5-bb8a-f849beeaca00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:48.619087 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca722d6-2996-44a5-bb8a-f849beeaca00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:48.626794 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca722d6-2996-44a5-bb8a-f849beeaca00 service cinder] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:48.651030 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1219/2448] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:19:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:19:49.627367 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094b69da-1da5-47de-9947-641c1537bdfe service cinder] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:49.644680 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094b69da-1da5-47de-9947-641c1537bdfe service cinder] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:49.654912 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c637ac-5e58-46c1-ae0f-c6c4f6541e7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:49.655545 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c637ac-5e58-46c1-ae0f-c6c4f6541e7a service cinder] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:49.655823 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c637ac-5e58-46c1-ae0f-c6c4f6541e7a service cinder] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:49.656097 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c637ac-5e58-46c1-ae0f-c6c4f6541e7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:49.691722 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c637ac-5e58-46c1-ae0f-c6c4f6541e7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:49.693475 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c637ac-5e58-46c1-ae0f-c6c4f6541e7a service cinder] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:49.711163 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1230/2449] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:19:49 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:19:49.874289 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ddd9e8-c3b3-4632-8614-a7a5a6db80a4 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:49.874785 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ddd9e8-c3b3-4632-8614-a7a5a6db80a4 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:49.874997 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ddd9e8-c3b3-4632-8614-a7a5a6db80a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:49.975661 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-87ddd9e8-c3b3-4632-8614-a7a5a6db80a4 None None] MFA Rules not processed for user `fca710369bd546599a21503e3fb505dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:19:49.978086 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-87ddd9e8-c3b3-4632-8614-a7a5a6db80a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:50.083891 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:19:50.103381 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1220/2450] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:19:49 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:19:50.104499 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c637ac-5e58-46c1-ae0f-c6c4f6541e7a service cinder] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:50.107349 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-63c637ac-5e58-46c1-ae0f-c6c4f6541e7a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:50.243444 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c637ac-5e58-46c1-ae0f-c6c4f6541e7a service cinder] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:50.254345 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3053049a-e069-4951-b5e5-b99e352ce71c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae643cfdcb164aa9b474747cd21aa116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:50.255277 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053049a-e069-4951-b5e5-b99e352ce71c service nova] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:50.255566 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053049a-e069-4951-b5e5-b99e352ce71c service nova] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:50.255900 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053049a-e069-4951-b5e5-b99e352ce71c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:50.258978 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-3053049a-e069-4951-b5e5-b99e352ce71c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:50.292574 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:19:50.365404 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3053049a-e069-4951-b5e5-b99e352ce71c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:50.367831 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3053049a-e069-4951-b5e5-b99e352ce71c service nova] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:50.383982 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1231/2451] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:19:50 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:19:52.422205 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9464f3-5139-4fa0-bcbb-a13e18b2c09e None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:52.423094 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9464f3-5139-4fa0-bcbb-a13e18b2c09e None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:52.423742 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9464f3-5139-4fa0-bcbb-a13e18b2c09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:52.454531 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-ce9464f3-5139-4fa0-bcbb-a13e18b2c09e None None] MFA Rules not processed for user `00917dba4a9f4d56be3ec74f90a65154`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:19:52.455868 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-ce9464f3-5139-4fa0-bcbb-a13e18b2c09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:52.473315 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1221/2452] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:19:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:19:54.244922 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3053049a-e069-4951-b5e5-b99e352ce71c service nova] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:54.348106 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3053049a-e069-4951-b5e5-b99e352ce71c service nova] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:54.353658 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2e105-0a41-4f90-8fb5-b95835e01248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:54.355118 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2e105-0a41-4f90-8fb5-b95835e01248 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:54.355396 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2e105-0a41-4f90-8fb5-b95835e01248 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:54.355678 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2e105-0a41-4f90-8fb5-b95835e01248 admin admin] PATH_INFO: `/v3/users/9c8f0dc722c746b081f7d0b83347acc9` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:54.356861 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a2e105-0a41-4f90-8fb5-b95835e01248 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c8f0dc722c746b081f7d0b83347acc9)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:54.360229 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a2e105-0a41-4f90-8fb5-b95835e01248 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:54.452930 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-28a2e105-0a41-4f90-8fb5-b95835e01248 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c8f0dc722c746b081f7d0b83347acc9'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:54.472541 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-28a2e105-0a41-4f90-8fb5-b95835e01248 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c8f0dc722c746b081f7d0b83347acc9'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:54.500027 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-28a2e105-0a41-4f90-8fb5-b95835e01248 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c8f0dc722c746b081f7d0b83347acc9'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:54.525711 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-28a2e105-0a41-4f90-8fb5-b95835e01248 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c8f0dc722c746b081f7d0b83347acc9'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:54.525711 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-28a2e105-0a41-4f90-8fb5-b95835e01248 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c8f0dc722c746b081f7d0b83347acc9'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:54.525711 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1232/2453] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:19:54 2022] DELETE /identity/v3/users/9c8f0dc722c746b081f7d0b83347acc9 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:19:54.527151 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde1b366-4414-48d4-a2cb-89426ab5ae9e None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:54.527615 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde1b366-4414-48d4-a2cb-89426ab5ae9e None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:54.527912 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde1b366-4414-48d4-a2cb-89426ab5ae9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:54.576388 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-cde1b366-4414-48d4-a2cb-89426ab5ae9e None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:19:54.580171 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-cde1b366-4414-48d4-a2cb-89426ab5ae9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:54.656691 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1222/2454] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:19:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:19:54.684886 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2e105-0a41-4f90-8fb5-b95835e01248 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:54.687196 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-28a2e105-0a41-4f90-8fb5-b95835e01248 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:54.771194 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:19:54.811116 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2e105-0a41-4f90-8fb5-b95835e01248 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:54.819068 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acff4d31-3cd9-4822-bfb7-fd6f1b425bda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:54.820027 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff4d31-3cd9-4822-bfb7-fd6f1b425bda service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:54.820543 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff4d31-3cd9-4822-bfb7-fd6f1b425bda service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:54.821313 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff4d31-3cd9-4822-bfb7-fd6f1b425bda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:54.835384 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acff4d31-3cd9-4822-bfb7-fd6f1b425bda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:54.837115 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acff4d31-3cd9-4822-bfb7-fd6f1b425bda service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:54.850001 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1233/2455] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:19:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:19:55.223539 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde1b366-4414-48d4-a2cb-89426ab5ae9e None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:55.225150 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-cde1b366-4414-48d4-a2cb-89426ab5ae9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:55.250690 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde1b366-4414-48d4-a2cb-89426ab5ae9e None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:55.265044 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c28553a-be77-4ed8-bd8d-37e17d7a7aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:55.265830 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c28553a-be77-4ed8-bd8d-37e17d7a7aa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:55.266101 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c28553a-be77-4ed8-bd8d-37e17d7a7aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:55.266372 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c28553a-be77-4ed8-bd8d-37e17d7a7aa5 admin admin] PATH_INFO: `/v3/projects/6de95c9f7d1948b895fb2830c4d5af4b` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:55.269064 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c28553a-be77-4ed8-bd8d-37e17d7a7aa5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6de95c9f7d1948b895fb2830c4d5af4b)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:55.271198 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c28553a-be77-4ed8-bd8d-37e17d7a7aa5 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:55.348449 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-8c28553a-be77-4ed8-bd8d-37e17d7a7aa5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6de95c9f7d1948b895fb2830c4d5af4b', 'request_id': 'req-8c28553a-be77-4ed8-bd8d-37e17d7a7aa5'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:55.361609 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-8c28553a-be77-4ed8-bd8d-37e17d7a7aa5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6de95c9f7d1948b895fb2830c4d5af4b', 'request_id': 'req-8c28553a-be77-4ed8-bd8d-37e17d7a7aa5'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:55.371337 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-8c28553a-be77-4ed8-bd8d-37e17d7a7aa5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6de95c9f7d1948b895fb2830c4d5af4b', 'request_id': 'req-8c28553a-be77-4ed8-bd8d-37e17d7a7aa5'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:55.407545 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-8c28553a-be77-4ed8-bd8d-37e17d7a7aa5 admin admin] The token cache is being invalidate because project 6de95c9f7d1948b895fb2830c4d5af4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81661) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:19:55.407839 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-8c28553a-be77-4ed8-bd8d-37e17d7a7aa5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:55.408630 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-8c28553a-be77-4ed8-bd8d-37e17d7a7aa5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:55.412483 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1223/2456] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:19:55 2022] DELETE /identity/v3/projects/6de95c9f7d1948b895fb2830c4d5af4b => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:19:55.414767 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acff4d31-3cd9-4822-bfb7-fd6f1b425bda service neutron] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:55.426011 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acff4d31-3cd9-4822-bfb7-fd6f1b425bda service neutron] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:55.497464 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f1ed09-830d-4dec-8bad-0478301c8f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:55.498160 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f1ed09-830d-4dec-8bad-0478301c8f80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:55.498627 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f1ed09-830d-4dec-8bad-0478301c8f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:55.498984 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f1ed09-830d-4dec-8bad-0478301c8f80 admin admin] PATH_INFO: `/v3/users/eaad5576abe6475cb110dcbb03d1e72c` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:55.500239 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f1ed09-830d-4dec-8bad-0478301c8f80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaad5576abe6475cb110dcbb03d1e72c)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:55.502488 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f1ed09-830d-4dec-8bad-0478301c8f80 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:55.587038 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-f6f1ed09-830d-4dec-8bad-0478301c8f80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaad5576abe6475cb110dcbb03d1e72c'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:55.602456 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-f6f1ed09-830d-4dec-8bad-0478301c8f80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaad5576abe6475cb110dcbb03d1e72c'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:55.611013 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-f6f1ed09-830d-4dec-8bad-0478301c8f80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaad5576abe6475cb110dcbb03d1e72c'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:55.626182 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-f6f1ed09-830d-4dec-8bad-0478301c8f80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaad5576abe6475cb110dcbb03d1e72c'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:55.626954 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-f6f1ed09-830d-4dec-8bad-0478301c8f80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaad5576abe6475cb110dcbb03d1e72c'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:55.645391 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1234/2457] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:19:55 2022] DELETE /identity/v3/users/eaad5576abe6475cb110dcbb03d1e72c => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:19:56.030879 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c28553a-be77-4ed8-bd8d-37e17d7a7aa5 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:56.033120 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-8c28553a-be77-4ed8-bd8d-37e17d7a7aa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:56.059949 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c28553a-be77-4ed8-bd8d-37e17d7a7aa5 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:56.140253 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e74fe27-c93a-47e3-a253-3e3435ef36ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:56.141013 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e74fe27-c93a-47e3-a253-3e3435ef36ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:56.141478 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e74fe27-c93a-47e3-a253-3e3435ef36ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:56.141947 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e74fe27-c93a-47e3-a253-3e3435ef36ee admin admin] PATH_INFO: `/v3/projects/3c6458a01d2f4cb093ef9c2c9ddd9e1c` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:56.143280 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e74fe27-c93a-47e3-a253-3e3435ef36ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c6458a01d2f4cb093ef9c2c9ddd9e1c)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:56.145413 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e74fe27-c93a-47e3-a253-3e3435ef36ee admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:56.182878 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:19:56.240782 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-6e74fe27-c93a-47e3-a253-3e3435ef36ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c6458a01d2f4cb093ef9c2c9ddd9e1c', 'request_id': 'req-6e74fe27-c93a-47e3-a253-3e3435ef36ee'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:56.255645 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-6e74fe27-c93a-47e3-a253-3e3435ef36ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c6458a01d2f4cb093ef9c2c9ddd9e1c', 'request_id': 'req-6e74fe27-c93a-47e3-a253-3e3435ef36ee'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:56.266294 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-6e74fe27-c93a-47e3-a253-3e3435ef36ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c6458a01d2f4cb093ef9c2c9ddd9e1c', 'request_id': 'req-6e74fe27-c93a-47e3-a253-3e3435ef36ee'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:56.279170 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-6e74fe27-c93a-47e3-a253-3e3435ef36ee admin admin] The token cache is being invalidate because project 3c6458a01d2f4cb093ef9c2c9ddd9e1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81661) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:19:56.279806 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-6e74fe27-c93a-47e3-a253-3e3435ef36ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:56.280920 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-6e74fe27-c93a-47e3-a253-3e3435ef36ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:56.282975 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1224/2458] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:19:56 2022] DELETE /identity/v3/projects/3c6458a01d2f4cb093ef9c2c9ddd9e1c => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:19:56.286781 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f1ed09-830d-4dec-8bad-0478301c8f80 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:56.289211 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-f6f1ed09-830d-4dec-8bad-0478301c8f80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:56.307688 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f1ed09-830d-4dec-8bad-0478301c8f80 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:56.381861 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b8e0d0-ee10-414e-8342-ab2b10cf3d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:56.382480 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b8e0d0-ee10-414e-8342-ab2b10cf3d75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:56.382773 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b8e0d0-ee10-414e-8342-ab2b10cf3d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:56.383047 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b8e0d0-ee10-414e-8342-ab2b10cf3d75 admin admin] PATH_INFO: `/v3/users/2f6acecfc8724b969ae5862e3519a80a` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:56.384517 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b8e0d0-ee10-414e-8342-ab2b10cf3d75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f6acecfc8724b969ae5862e3519a80a)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:56.387291 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b8e0d0-ee10-414e-8342-ab2b10cf3d75 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:56.389275 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:19:56.443999 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-c9b8e0d0-ee10-414e-8342-ab2b10cf3d75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f6acecfc8724b969ae5862e3519a80a'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:56.456597 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-c9b8e0d0-ee10-414e-8342-ab2b10cf3d75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f6acecfc8724b969ae5862e3519a80a'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:56.464882 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-c9b8e0d0-ee10-414e-8342-ab2b10cf3d75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f6acecfc8724b969ae5862e3519a80a'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:56.472730 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-c9b8e0d0-ee10-414e-8342-ab2b10cf3d75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f6acecfc8724b969ae5862e3519a80a'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:56.473537 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-c9b8e0d0-ee10-414e-8342-ab2b10cf3d75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f6acecfc8724b969ae5862e3519a80a'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:56.480603 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1235/2459] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:19:56 2022] DELETE /identity/v3/users/2f6acecfc8724b969ae5862e3519a80a => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:19:56.819038 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e74fe27-c93a-47e3-a253-3e3435ef36ee admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:56.820873 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-6e74fe27-c93a-47e3-a253-3e3435ef36ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:56.835844 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e74fe27-c93a-47e3-a253-3e3435ef36ee admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:56.898212 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a60eb9b-0cbd-4eeb-9aca-1ea639a6a7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:56.898853 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a60eb9b-0cbd-4eeb-9aca-1ea639a6a7d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:56.899715 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a60eb9b-0cbd-4eeb-9aca-1ea639a6a7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:56.900011 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a60eb9b-0cbd-4eeb-9aca-1ea639a6a7d6 admin admin] PATH_INFO: `/v3/projects/39aec913db2f48f68f4121749cffe2f8` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:56.901589 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a60eb9b-0cbd-4eeb-9aca-1ea639a6a7d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39aec913db2f48f68f4121749cffe2f8)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:56.903631 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a60eb9b-0cbd-4eeb-9aca-1ea639a6a7d6 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:56.964401 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-0a60eb9b-0cbd-4eeb-9aca-1ea639a6a7d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39aec913db2f48f68f4121749cffe2f8', 'request_id': 'req-0a60eb9b-0cbd-4eeb-9aca-1ea639a6a7d6'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:56.973068 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-0a60eb9b-0cbd-4eeb-9aca-1ea639a6a7d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39aec913db2f48f68f4121749cffe2f8', 'request_id': 'req-0a60eb9b-0cbd-4eeb-9aca-1ea639a6a7d6'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:56.980577 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-0a60eb9b-0cbd-4eeb-9aca-1ea639a6a7d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39aec913db2f48f68f4121749cffe2f8', 'request_id': 'req-0a60eb9b-0cbd-4eeb-9aca-1ea639a6a7d6'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:56.991227 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-0a60eb9b-0cbd-4eeb-9aca-1ea639a6a7d6 admin admin] The token cache is being invalidate because project 39aec913db2f48f68f4121749cffe2f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81661) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:19:56.991509 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-0a60eb9b-0cbd-4eeb-9aca-1ea639a6a7d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:56.992249 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-0a60eb9b-0cbd-4eeb-9aca-1ea639a6a7d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:19:56.993725 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1225/2460] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:19:56 2022] DELETE /identity/v3/projects/39aec913db2f48f68f4121749cffe2f8 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:19:56.998777 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d90be4-6a6e-4b8e-b46d-621289e8fc28 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:56.999071 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d90be4-6a6e-4b8e-b46d-621289e8fc28 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:56.999347 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d90be4-6a6e-4b8e-b46d-621289e8fc28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:57.021081 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-39d90be4-6a6e-4b8e-b46d-621289e8fc28 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:19:57.022372 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-39d90be4-6a6e-4b8e-b46d-621289e8fc28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:57.093726 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1236/2461] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:19:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:19:57.095475 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d05f8-0b77-480b-b8ce-f936d9268e6a None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:57.095936 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d05f8-0b77-480b-b8ce-f936d9268e6a None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:57.096357 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d05f8-0b77-480b-b8ce-f936d9268e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:57.120780 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-b89d05f8-0b77-480b-b8ce-f936d9268e6a None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:19:57.122335 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-b89d05f8-0b77-480b-b8ce-f936d9268e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:57.131297 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1226/2462] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:19:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:19:57.135834 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d90be4-6a6e-4b8e-b46d-621289e8fc28 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:57.145931 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d90be4-6a6e-4b8e-b46d-621289e8fc28 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:57.150677 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c5d652-e28e-442f-b9b0-add01a178650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:57.151294 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5d652-e28e-442f-b9b0-add01a178650 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:57.151558 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5d652-e28e-442f-b9b0-add01a178650 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:57.151840 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5d652-e28e-442f-b9b0-add01a178650 admin admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:57.152633 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c5d652-e28e-442f-b9b0-add01a178650 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:19:57.152948 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c5d652-e28e-442f-b9b0-add01a178650 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:57.154576 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c5d652-e28e-442f-b9b0-add01a178650 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:57.167270 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1237/2463] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:19:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:19:57.171767 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d05f8-0b77-480b-b8ce-f936d9268e6a None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:57.180980 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d05f8-0b77-480b-b8ce-f936d9268e6a None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:57.186550 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baaa503-dcd7-4830-972f-e2732685d04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:57.187158 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baaa503-dcd7-4830-972f-e2732685d04b admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:57.187435 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baaa503-dcd7-4830-972f-e2732685d04b admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:57.187707 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baaa503-dcd7-4830-972f-e2732685d04b admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:57.188471 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baaa503-dcd7-4830-972f-e2732685d04b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:57.190382 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baaa503-dcd7-4830-972f-e2732685d04b admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:57.219078 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1227/2464] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:19:57 2022] POST /identity/v3/projects => generated 367 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:19:57.223296 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c5d652-e28e-442f-b9b0-add01a178650 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:57.234228 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c5d652-e28e-442f-b9b0-add01a178650 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:57.298684 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b43cca-a8ab-466c-9687-29948407ee6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:57.299311 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b43cca-a8ab-466c-9687-29948407ee6d admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:57.299720 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b43cca-a8ab-466c-9687-29948407ee6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:57.299955 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b43cca-a8ab-466c-9687-29948407ee6d admin admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:57.300897 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b43cca-a8ab-466c-9687-29948407ee6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:57.302731 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b43cca-a8ab-466c-9687-29948407ee6d admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:57.326021 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:19:57.337614 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1238/2465] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:19:57 2022] POST /identity/v3/users => generated 284 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 01:19:57.342372 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baaa503-dcd7-4830-972f-e2732685d04b admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:57.352803 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baaa503-dcd7-4830-972f-e2732685d04b admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:57.357930 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d838511-3e9c-46fd-a205-8b79aa54c5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:57.358524 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d838511-3e9c-46fd-a205-8b79aa54c5b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:57.358820 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d838511-3e9c-46fd-a205-8b79aa54c5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:57.359122 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d838511-3e9c-46fd-a205-8b79aa54c5b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:57.359711 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d838511-3e9c-46fd-a205-8b79aa54c5b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:57.361349 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d838511-3e9c-46fd-a205-8b79aa54c5b0 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:57.367905 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:19:57.375494 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1228/2466] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:19:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:19:57.382916 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b43cca-a8ab-466c-9687-29948407ee6d admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:57.394732 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b43cca-a8ab-466c-9687-29948407ee6d admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:57.400346 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d29196d-1938-4269-bf52-020f12de0cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:57.401605 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d29196d-1938-4269-bf52-020f12de0cf8 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:57.402442 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d29196d-1938-4269-bf52-020f12de0cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:57.402992 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d29196d-1938-4269-bf52-020f12de0cf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:57.403762 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d29196d-1938-4269-bf52-020f12de0cf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:57.405909 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d29196d-1938-4269-bf52-020f12de0cf8 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:57.418094 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1239/2467] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:19:57 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:19:57.426746 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d838511-3e9c-46fd-a205-8b79aa54c5b0 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:57.442522 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d838511-3e9c-46fd-a205-8b79aa54c5b0 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:57.448484 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bdf167-14e3-4c5a-843e-681076267241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:57.449157 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bdf167-14e3-4c5a-843e-681076267241 admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:57.449440 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bdf167-14e3-4c5a-843e-681076267241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:57.449720 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bdf167-14e3-4c5a-843e-681076267241 admin admin] PATH_INFO: `/v3/projects/2cc3055e3792423b96fbd6f085f0bb87/users/b20b182c6eec459584b740d443ef5b96/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:57.464810 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bdf167-14e3-4c5a-843e-681076267241 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cc3055e3792423b96fbd6f085f0bb87, user_id=b20b182c6eec459584b740d443ef5b96, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:57.467494 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bdf167-14e3-4c5a-843e-681076267241 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:57.482894 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1229/2468] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:19:57 2022] PUT /identity/v3/projects/2cc3055e3792423b96fbd6f085f0bb87/users/b20b182c6eec459584b740d443ef5b96/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:19:57.493101 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa7e4f-2cff-405d-bd22-ce3b55ae055d None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:57.493599 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa7e4f-2cff-405d-bd22-ce3b55ae055d None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:57.493722 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa7e4f-2cff-405d-bd22-ce3b55ae055d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:57.522793 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-94fa7e4f-2cff-405d-bd22-ce3b55ae055d None None] MFA Rules not processed for user `b20b182c6eec459584b740d443ef5b96`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:19:57.524035 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-94fa7e4f-2cff-405d-bd22-ce3b55ae055d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:57.592415 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1240/2469] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:19:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:19:57.604607 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bdf167-14e3-4c5a-843e-681076267241 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:19:57.606393 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-08bdf167-14e3-4c5a-843e-681076267241 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:19:57.664539 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bdf167-14e3-4c5a-843e-681076267241 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:19:57.669031 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffd1912-815d-4c45-ae7f-4c2baf0ed881 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1acfa7a09d34e2a9045c3f3fbea0f52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:19:57.669628 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd1912-815d-4c45-ae7f-4c2baf0ed881 service glance] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:19:57.669905 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd1912-815d-4c45-ae7f-4c2baf0ed881 service glance] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:19:57.670177 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd1912-815d-4c45-ae7f-4c2baf0ed881 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:19:57.679135 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffd1912-815d-4c45-ae7f-4c2baf0ed881 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:19:57.680566 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffd1912-815d-4c45-ae7f-4c2baf0ed881 service glance] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:19:57.691698 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1230/2470] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:19:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:20:06.200574 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fa7e4f-2cff-405d-bd22-ce3b55ae055d None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:20:06.202048 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-94fa7e4f-2cff-405d-bd22-ce3b55ae055d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:20:06.267485 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fa7e4f-2cff-405d-bd22-ce3b55ae055d None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:20:06.271441 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d90d556-6da6-4458-81b5-5d1c80da1c1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:20:06.272071 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d90d556-6da6-4458-81b5-5d1c80da1c1a service cinder] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:20:06.272379 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d90d556-6da6-4458-81b5-5d1c80da1c1a service cinder] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:20:06.272655 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d90d556-6da6-4458-81b5-5d1c80da1c1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:20:06.274630 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-6d90d556-6da6-4458-81b5-5d1c80da1c1a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:20:06.334494 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d90d556-6da6-4458-81b5-5d1c80da1c1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:20:06.335929 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d90d556-6da6-4458-81b5-5d1c80da1c1a service cinder] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:20:06.345570 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1241/2471] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:20:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:20:07.686906 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=37,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:20:09.853072 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff21c853-dac3-44d7-be18-35c083bb9134 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:20:09.854066 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff21c853-dac3-44d7-be18-35c083bb9134 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:20:09.854066 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff21c853-dac3-44d7-be18-35c083bb9134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:20:09.873418 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-ff21c853-dac3-44d7-be18-35c083bb9134 None None] MFA Rules not processed for user `de75fd7d7721433684cafa2e6d3faaae`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:20:09.874635 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-ff21c853-dac3-44d7-be18-35c083bb9134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:20:09.936367 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1231/2472] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Sep 8 01:20:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:20:09.950616 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d90d556-6da6-4458-81b5-5d1c80da1c1a service cinder] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:20:09.952007 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-6d90d556-6da6-4458-81b5-5d1c80da1c1a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:20:09.986349 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:20:10.030749 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d90d556-6da6-4458-81b5-5d1c80da1c1a service cinder] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:20:10.036475 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79ced84-31e4-44bd-8f13-516fce9a1522 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8add26872fe6479aa9d7de155a17195a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:20:10.037392 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ced84-31e4-44bd-8f13-516fce9a1522 service swift] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:20:10.037858 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ced84-31e4-44bd-8f13-516fce9a1522 service swift] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:20:10.038300 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ced84-31e4-44bd-8f13-516fce9a1522 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:20:10.048104 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79ced84-31e4-44bd-8f13-516fce9a1522 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:20:10.049791 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79ced84-31e4-44bd-8f13-516fce9a1522 service swift] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:20:10.058356 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1242/2473] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Sep 8 01:20:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 01:20:13.715987 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff21c853-dac3-44d7-be18-35c083bb9134 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:20:13.717584 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-ff21c853-dac3-44d7-be18-35c083bb9134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:20:13.793602 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff21c853-dac3-44d7-be18-35c083bb9134 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:20:13.797430 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa8575d-2a44-4cd8-8c1f-c599d2f40221 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:20:13.798049 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa8575d-2a44-4cd8-8c1f-c599d2f40221 service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:20:13.798331 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa8575d-2a44-4cd8-8c1f-c599d2f40221 service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:20:13.798612 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa8575d-2a44-4cd8-8c1f-c599d2f40221 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:20:13.800442 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-0aa8575d-2a44-4cd8-8c1f-c599d2f40221 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:20:13.812704 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa8575d-2a44-4cd8-8c1f-c599d2f40221 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:20:13.814367 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa8575d-2a44-4cd8-8c1f-c599d2f40221 service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:20:13.823713 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1232/2474] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:20:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:20:20.054874 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=19 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:20:23.819124 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=51 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:20:30.954173 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc462918-9c19-4fd1-9dfb-95899042ecd7 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:20:30.954684 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc462918-9c19-4fd1-9dfb-95899042ecd7 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:20:30.954684 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc462918-9c19-4fd1-9dfb-95899042ecd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:20:30.974434 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-cc462918-9c19-4fd1-9dfb-95899042ecd7 None None] MFA Rules not processed for user `83fea9762aaf4766a23f1774ff1411fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:20:30.975642 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-cc462918-9c19-4fd1-9dfb-95899042ecd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:20:30.983172 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1243/2475] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:20:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:20:33.304234 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b245f379-bbfb-48d2-a33d-d3a7def9b174 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:20:33.304786 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b245f379-bbfb-48d2-a33d-d3a7def9b174 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:20:33.304786 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b245f379-bbfb-48d2-a33d-d3a7def9b174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:20:33.327035 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-b245f379-bbfb-48d2-a33d-d3a7def9b174 None None] MFA Rules not processed for user `de75fd7d7721433684cafa2e6d3faaae`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:20:33.328733 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-b245f379-bbfb-48d2-a33d-d3a7def9b174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:20:33.336276 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1233/2476] 10.222.0.42 () {58 vars in 967 bytes} [Thu Sep 8 01:20:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:20:33.350125 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc462918-9c19-4fd1-9dfb-95899042ecd7 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:20:33.363002 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc462918-9c19-4fd1-9dfb-95899042ecd7 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:20:33.368110 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb58c45-2cbc-48e8-a6bc-2c86c67d7a6e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8add26872fe6479aa9d7de155a17195a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:20:33.368749 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb58c45-2cbc-48e8-a6bc-2c86c67d7a6e service swift] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:20:33.369032 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb58c45-2cbc-48e8-a6bc-2c86c67d7a6e service swift] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:20:33.369304 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb58c45-2cbc-48e8-a6bc-2c86c67d7a6e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:20:33.377987 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb58c45-2cbc-48e8-a6bc-2c86c67d7a6e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:20:33.379584 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb58c45-2cbc-48e8-a6bc-2c86c67d7a6e service swift] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:20:33.389854 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1244/2477] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Sep 8 01:20:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 01:20:35.442686 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2de833-b12f-4c7c-9a16-e08938d3101a None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:20:35.444314 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2de833-b12f-4c7c-9a16-e08938d3101a None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:20:35.444835 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2de833-b12f-4c7c-9a16-e08938d3101a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:20:35.466244 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-1e2de833-b12f-4c7c-9a16-e08938d3101a None None] MFA Rules not processed for user `744fe07ea5644157bdbf11d81adbab89`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:20:35.467748 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-1e2de833-b12f-4c7c-9a16-e08938d3101a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:20:35.532340 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1234/2478] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:20:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:20:37.139239 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb58c45-2cbc-48e8-a6bc-2c86c67d7a6e service swift] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:20:37.221373 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb58c45-2cbc-48e8-a6bc-2c86c67d7a6e service swift] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:20:37.228053 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349ccdda-b0ef-478e-809c-0f0f9500f011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:20:37.228814 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ccdda-b0ef-478e-809c-0f0f9500f011 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:20:37.229319 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ccdda-b0ef-478e-809c-0f0f9500f011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:20:37.229874 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ccdda-b0ef-478e-809c-0f0f9500f011 admin admin] PATH_INFO: `/v3/users/b20b182c6eec459584b740d443ef5b96` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:20:37.232279 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349ccdda-b0ef-478e-809c-0f0f9500f011 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b20b182c6eec459584b740d443ef5b96)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:20:37.237364 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349ccdda-b0ef-478e-809c-0f0f9500f011 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:20:37.302502 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-349ccdda-b0ef-478e-809c-0f0f9500f011 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b20b182c6eec459584b740d443ef5b96'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:20:37.322170 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-349ccdda-b0ef-478e-809c-0f0f9500f011 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b20b182c6eec459584b740d443ef5b96'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:20:37.332783 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-349ccdda-b0ef-478e-809c-0f0f9500f011 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b20b182c6eec459584b740d443ef5b96'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:20:37.342520 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-349ccdda-b0ef-478e-809c-0f0f9500f011 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b20b182c6eec459584b740d443ef5b96'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:20:37.343441 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-349ccdda-b0ef-478e-809c-0f0f9500f011 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b20b182c6eec459584b740d443ef5b96'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:20:37.351839 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1245/2479] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:20:37 2022] DELETE /identity/v3/users/b20b182c6eec459584b740d443ef5b96 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:20:37.356838 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d19de-954e-442a-bfb6-f83f454b5701 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:20:37.357305 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d19de-954e-442a-bfb6-f83f454b5701 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:20:37.357408 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d19de-954e-442a-bfb6-f83f454b5701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:20:37.405633 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-8c8d19de-954e-442a-bfb6-f83f454b5701 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:20:37.406924 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-8c8d19de-954e-442a-bfb6-f83f454b5701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:20:37.473386 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1235/2480] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:20:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:20:37.489023 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349ccdda-b0ef-478e-809c-0f0f9500f011 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:20:37.491046 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-349ccdda-b0ef-478e-809c-0f0f9500f011 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:20:37.574499 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349ccdda-b0ef-478e-809c-0f0f9500f011 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:20:37.578397 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ee3ad0-80fb-4df1-be03-b940a00d145b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:20:37.578968 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee3ad0-80fb-4df1-be03-b940a00d145b service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:20:37.579245 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee3ad0-80fb-4df1-be03-b940a00d145b service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:20:37.579518 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee3ad0-80fb-4df1-be03-b940a00d145b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:20:37.590327 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ee3ad0-80fb-4df1-be03-b940a00d145b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:20:37.591764 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ee3ad0-80fb-4df1-be03-b940a00d145b service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:20:37.601946 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1246/2481] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:20:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:20:38.038139 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d19de-954e-442a-bfb6-f83f454b5701 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:20:38.039572 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-8c8d19de-954e-442a-bfb6-f83f454b5701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:20:38.052432 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d19de-954e-442a-bfb6-f83f454b5701 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:20:38.058030 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d0ec1f-1e7e-4acc-80b9-019ec609d216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:20:38.058773 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d0ec1f-1e7e-4acc-80b9-019ec609d216 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:20:38.059110 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d0ec1f-1e7e-4acc-80b9-019ec609d216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:20:38.059450 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d0ec1f-1e7e-4acc-80b9-019ec609d216 admin admin] PATH_INFO: `/v3/projects/2cc3055e3792423b96fbd6f085f0bb87` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:20:38.060738 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d0ec1f-1e7e-4acc-80b9-019ec609d216 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cc3055e3792423b96fbd6f085f0bb87)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:20:38.062755 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d0ec1f-1e7e-4acc-80b9-019ec609d216 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:20:38.118776 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-f4d0ec1f-1e7e-4acc-80b9-019ec609d216 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cc3055e3792423b96fbd6f085f0bb87', 'request_id': 'req-f4d0ec1f-1e7e-4acc-80b9-019ec609d216'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:20:38.127695 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-f4d0ec1f-1e7e-4acc-80b9-019ec609d216 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cc3055e3792423b96fbd6f085f0bb87', 'request_id': 'req-f4d0ec1f-1e7e-4acc-80b9-019ec609d216'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:20:38.135072 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-f4d0ec1f-1e7e-4acc-80b9-019ec609d216 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cc3055e3792423b96fbd6f085f0bb87', 'request_id': 'req-f4d0ec1f-1e7e-4acc-80b9-019ec609d216'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:20:38.147779 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-f4d0ec1f-1e7e-4acc-80b9-019ec609d216 admin admin] The token cache is being invalidate because project 2cc3055e3792423b96fbd6f085f0bb87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81661) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:20:38.148068 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-f4d0ec1f-1e7e-4acc-80b9-019ec609d216 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:20:38.148971 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-f4d0ec1f-1e7e-4acc-80b9-019ec609d216 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:20:38.150502 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1236/2482] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:20:38 2022] DELETE /identity/v3/projects/2cc3055e3792423b96fbd6f085f0bb87 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:20:38.156874 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3ceb0e-4dcd-49ce-bd7f-70310a282fd9 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:20:38.157738 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3ceb0e-4dcd-49ce-bd7f-70310a282fd9 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:20:38.158076 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3ceb0e-4dcd-49ce-bd7f-70310a282fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:20:38.167298 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:20:38.182889 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-0f3ceb0e-4dcd-49ce-bd7f-70310a282fd9 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:20:38.184468 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-0f3ceb0e-4dcd-49ce-bd7f-70310a282fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:20:38.251822 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1247/2483] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:20:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:20:38.264181 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b33866-1c21-4e71-a3d3-27a50a32b796 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:20:38.268257 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b33866-1c21-4e71-a3d3-27a50a32b796 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:20:38.268574 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b33866-1c21-4e71-a3d3-27a50a32b796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:20:38.299097 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-c8b33866-1c21-4e71-a3d3-27a50a32b796 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:20:38.300759 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-c8b33866-1c21-4e71-a3d3-27a50a32b796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:20:38.313233 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1237/2484] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:20:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:20:38.319712 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3ceb0e-4dcd-49ce-bd7f-70310a282fd9 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:20:38.342295 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3ceb0e-4dcd-49ce-bd7f-70310a282fd9 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:20:38.361407 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34838a43-9de6-41b6-a713-de02ce20859f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:20:38.362901 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34838a43-9de6-41b6-a713-de02ce20859f admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:20:38.363403 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34838a43-9de6-41b6-a713-de02ce20859f admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:20:38.368244 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34838a43-9de6-41b6-a713-de02ce20859f admin admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:20:38.371607 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34838a43-9de6-41b6-a713-de02ce20859f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:20:38.372429 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34838a43-9de6-41b6-a713-de02ce20859f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:20:38.386474 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34838a43-9de6-41b6-a713-de02ce20859f admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:20:38.407164 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1248/2485] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:20:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:20:38.411954 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b33866-1c21-4e71-a3d3-27a50a32b796 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:20:38.423857 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b33866-1c21-4e71-a3d3-27a50a32b796 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:20:38.428810 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5276da8-2164-4b2d-ae69-c0a8e7925e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:20:38.429447 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5276da8-2164-4b2d-ae69-c0a8e7925e3e admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:20:38.429722 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5276da8-2164-4b2d-ae69-c0a8e7925e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:20:38.430186 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5276da8-2164-4b2d-ae69-c0a8e7925e3e admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:20:38.430818 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5276da8-2164-4b2d-ae69-c0a8e7925e3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:20:38.433520 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5276da8-2164-4b2d-ae69-c0a8e7925e3e admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:20:38.452557 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:20:38.470655 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1238/2486] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:20:38 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:20:38.475103 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34838a43-9de6-41b6-a713-de02ce20859f admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:20:38.486330 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34838a43-9de6-41b6-a713-de02ce20859f admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:20:38.554849 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f56212-3f73-4e9c-b8e1-605c86e1b287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:20:38.555459 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f56212-3f73-4e9c-b8e1-605c86e1b287 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:20:38.555810 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f56212-3f73-4e9c-b8e1-605c86e1b287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:20:38.556110 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f56212-3f73-4e9c-b8e1-605c86e1b287 admin admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:20:38.556821 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f56212-3f73-4e9c-b8e1-605c86e1b287 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:20:38.559508 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f56212-3f73-4e9c-b8e1-605c86e1b287 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:20:38.584245 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1249/2487] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:20:38 2022] POST /identity/v3/users => generated 287 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:20:38.586972 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5276da8-2164-4b2d-ae69-c0a8e7925e3e admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:20:38.598267 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5276da8-2164-4b2d-ae69-c0a8e7925e3e admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:20:38.603393 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b488b5a-8bb6-4827-9c61-f42130479ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:20:38.604116 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b488b5a-8bb6-4827-9c61-f42130479ab7 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:20:38.605040 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b488b5a-8bb6-4827-9c61-f42130479ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:20:38.605812 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b488b5a-8bb6-4827-9c61-f42130479ab7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:20:38.606640 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b488b5a-8bb6-4827-9c61-f42130479ab7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:20:38.608345 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b488b5a-8bb6-4827-9c61-f42130479ab7 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:20:38.618803 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1239/2488] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:20:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:20:38.625118 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f56212-3f73-4e9c-b8e1-605c86e1b287 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:20:38.637523 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f56212-3f73-4e9c-b8e1-605c86e1b287 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:20:38.642537 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb28b7a0-022f-4508-aad2-d8277c22a1dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:20:38.643284 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28b7a0-022f-4508-aad2-d8277c22a1dd admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:20:38.643534 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28b7a0-022f-4508-aad2-d8277c22a1dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:20:38.643904 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28b7a0-022f-4508-aad2-d8277c22a1dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:20:38.644574 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb28b7a0-022f-4508-aad2-d8277c22a1dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:20:38.646164 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb28b7a0-022f-4508-aad2-d8277c22a1dd admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:20:38.660855 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1250/2489] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:20:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:20:38.662246 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b488b5a-8bb6-4827-9c61-f42130479ab7 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:20:38.674121 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b488b5a-8bb6-4827-9c61-f42130479ab7 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:20:38.679174 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdf46d1-05ee-4a00-a3e2-0c4e1c3edaaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:20:38.679919 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf46d1-05ee-4a00-a3e2-0c4e1c3edaaf admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:20:38.680322 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf46d1-05ee-4a00-a3e2-0c4e1c3edaaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:20:38.680657 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf46d1-05ee-4a00-a3e2-0c4e1c3edaaf admin admin] PATH_INFO: `/v3/projects/61a0e556dd0242b09ff083c61bebb335/users/4fda6370eff34bab97d00d716c26f471/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:20:38.700271 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdf46d1-05ee-4a00-a3e2-0c4e1c3edaaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=61a0e556dd0242b09ff083c61bebb335, user_id=4fda6370eff34bab97d00d716c26f471, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:20:38.703011 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdf46d1-05ee-4a00-a3e2-0c4e1c3edaaf admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:20:38.717366 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1240/2490] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:20:38 2022] PUT /identity/v3/projects/61a0e556dd0242b09ff083c61bebb335/users/4fda6370eff34bab97d00d716c26f471/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:20:38.727985 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be5fbad-e895-4409-9da9-43e4b9330cce None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:20:38.728378 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be5fbad-e895-4409-9da9-43e4b9330cce None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:20:38.728661 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be5fbad-e895-4409-9da9-43e4b9330cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:20:38.758622 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-1be5fbad-e895-4409-9da9-43e4b9330cce None None] MFA Rules not processed for user `4fda6370eff34bab97d00d716c26f471`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:20:38.759859 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-1be5fbad-e895-4409-9da9-43e4b9330cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:20:38.826991 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:20:38.834989 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1251/2491] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:20:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:20:38.847108 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdf46d1-05ee-4a00-a3e2-0c4e1c3edaaf admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:20:38.848901 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-1bdf46d1-05ee-4a00-a3e2-0c4e1c3edaaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:20:38.913189 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdf46d1-05ee-4a00-a3e2-0c4e1c3edaaf admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:20:38.917270 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937cda4e-fc8a-4e2a-8638-79b029c0013d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:20:38.917969 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937cda4e-fc8a-4e2a-8638-79b029c0013d service cinder] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:20:38.918253 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937cda4e-fc8a-4e2a-8638-79b029c0013d service cinder] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:20:38.918527 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937cda4e-fc8a-4e2a-8638-79b029c0013d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:20:38.928413 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937cda4e-fc8a-4e2a-8638-79b029c0013d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:20:38.929868 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937cda4e-fc8a-4e2a-8638-79b029c0013d service cinder] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:20:38.938927 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1241/2492] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:20:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:20:38.965829 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be5fbad-e895-4409-9da9-43e4b9330cce None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:20:39.033787 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be5fbad-e895-4409-9da9-43e4b9330cce None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:20:39.038540 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c1f8c0-9586-4cea-8c85-b85254599fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:20:39.039178 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c1f8c0-9586-4cea-8c85-b85254599fa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:20:39.039452 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c1f8c0-9586-4cea-8c85-b85254599fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:20:39.039730 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c1f8c0-9586-4cea-8c85-b85254599fa2 admin admin] PATH_INFO: `/v3/users/4fda6370eff34bab97d00d716c26f471` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:20:39.040915 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c1f8c0-9586-4cea-8c85-b85254599fa2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fda6370eff34bab97d00d716c26f471)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:20:39.042895 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c1f8c0-9586-4cea-8c85-b85254599fa2 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:20:39.099626 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-82c1f8c0-9586-4cea-8c85-b85254599fa2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fda6370eff34bab97d00d716c26f471'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:20:39.118880 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-82c1f8c0-9586-4cea-8c85-b85254599fa2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fda6370eff34bab97d00d716c26f471'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:20:39.138923 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-82c1f8c0-9586-4cea-8c85-b85254599fa2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fda6370eff34bab97d00d716c26f471'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:20:39.153276 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-82c1f8c0-9586-4cea-8c85-b85254599fa2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fda6370eff34bab97d00d716c26f471'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:20:39.154018 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-82c1f8c0-9586-4cea-8c85-b85254599fa2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fda6370eff34bab97d00d716c26f471'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:20:39.163105 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1252/2493] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:20:38 2022] DELETE /identity/v3/users/4fda6370eff34bab97d00d716c26f471 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:20:39.167659 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65529b2a-b484-495f-af2d-02352318b614 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:20:39.168379 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65529b2a-b484-495f-af2d-02352318b614 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:20:39.168379 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65529b2a-b484-495f-af2d-02352318b614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:20:39.194523 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-65529b2a-b484-495f-af2d-02352318b614 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:20:39.195891 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-65529b2a-b484-495f-af2d-02352318b614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:20:39.272852 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1242/2494] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:20:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:20:39.283075 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c1f8c0-9586-4cea-8c85-b85254599fa2 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:20:39.284544 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-82c1f8c0-9586-4cea-8c85-b85254599fa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:20:39.355909 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c1f8c0-9586-4cea-8c85-b85254599fa2 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:20:39.359533 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a932f-6bda-408d-b8cf-69c10c76dd7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:20:39.360097 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a932f-6bda-408d-b8cf-69c10c76dd7f service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:20:39.360417 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a932f-6bda-408d-b8cf-69c10c76dd7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:20:39.360805 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a932f-6bda-408d-b8cf-69c10c76dd7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:20:39.370969 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79a932f-6bda-408d-b8cf-69c10c76dd7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:20:39.372434 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79a932f-6bda-408d-b8cf-69c10c76dd7f service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:20:39.382163 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1253/2495] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:20:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 01:20:39.873630 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65529b2a-b484-495f-af2d-02352318b614 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:20:39.875136 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-65529b2a-b484-495f-af2d-02352318b614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:20:39.888413 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65529b2a-b484-495f-af2d-02352318b614 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:20:39.893837 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261f4b48-1d49-4b2f-9035-c0152ed309ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:20:39.894721 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f4b48-1d49-4b2f-9035-c0152ed309ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:20:39.895224 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f4b48-1d49-4b2f-9035-c0152ed309ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:20:39.895698 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f4b48-1d49-4b2f-9035-c0152ed309ac admin admin] PATH_INFO: `/v3/projects/61a0e556dd0242b09ff083c61bebb335` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:20:39.897308 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261f4b48-1d49-4b2f-9035-c0152ed309ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=61a0e556dd0242b09ff083c61bebb335)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:20:39.899681 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261f4b48-1d49-4b2f-9035-c0152ed309ac admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:20:39.969386 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-261f4b48-1d49-4b2f-9035-c0152ed309ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61a0e556dd0242b09ff083c61bebb335', 'request_id': 'req-261f4b48-1d49-4b2f-9035-c0152ed309ac'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:20:39.981383 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-261f4b48-1d49-4b2f-9035-c0152ed309ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61a0e556dd0242b09ff083c61bebb335', 'request_id': 'req-261f4b48-1d49-4b2f-9035-c0152ed309ac'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:20:39.982841 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:INSERT=3,keystone:SELECT=92,keystone:DELETE=5 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:20:39.989831 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-261f4b48-1d49-4b2f-9035-c0152ed309ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61a0e556dd0242b09ff083c61bebb335', 'request_id': 'req-261f4b48-1d49-4b2f-9035-c0152ed309ac'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:20:40.004335 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-261f4b48-1d49-4b2f-9035-c0152ed309ac admin admin] The token cache is being invalidate because project 61a0e556dd0242b09ff083c61bebb335 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81661) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:20:40.004717 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-261f4b48-1d49-4b2f-9035-c0152ed309ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:20:40.005721 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-261f4b48-1d49-4b2f-9035-c0152ed309ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:20:40.007345 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1243/2496] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:20:39 2022] DELETE /identity/v3/projects/61a0e556dd0242b09ff083c61bebb335 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:20:46.698037 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a932f-6bda-408d-b8cf-69c10c76dd7f service neutron] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:20:46.708071 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a932f-6bda-408d-b8cf-69c10c76dd7f service neutron] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:20:46.728535 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:20:46.774389 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3103e73-90c9-4fe1-8aeb-a44dce464681 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:20:46.775011 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3103e73-90c9-4fe1-8aeb-a44dce464681 service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:20:46.775295 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3103e73-90c9-4fe1-8aeb-a44dce464681 service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:20:46.776106 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3103e73-90c9-4fe1-8aeb-a44dce464681 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:20:46.778639 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-c3103e73-90c9-4fe1-8aeb-a44dce464681 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:20:46.849402 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3103e73-90c9-4fe1-8aeb-a44dce464681 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:20:46.850848 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3103e73-90c9-4fe1-8aeb-a44dce464681 service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:20:46.861318 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1254/2497] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:20:46 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:20:50.001046 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=5,keystone:DELETE=2 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:20:56.856750 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=41 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:20:59.473272 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261f4b48-1d49-4b2f-9035-c0152ed309ac admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:20:59.474717 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-261f4b48-1d49-4b2f-9035-c0152ed309ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:20:59.488100 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261f4b48-1d49-4b2f-9035-c0152ed309ac admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:20:59.491881 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd52ed-6480-4f3e-bc98-42a2ae660b31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:20:59.492544 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd52ed-6480-4f3e-bc98-42a2ae660b31 service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:20:59.492823 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd52ed-6480-4f3e-bc98-42a2ae660b31 service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:20:59.493097 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd52ed-6480-4f3e-bc98-42a2ae660b31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:20:59.494942 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-9efd52ed-6480-4f3e-bc98-42a2ae660b31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:20:59.570665 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd52ed-6480-4f3e-bc98-42a2ae660b31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:20:59.572228 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd52ed-6480-4f3e-bc98-42a2ae660b31 service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:20:59.583453 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1244/2498] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:20:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:21:02.729513 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3103e73-90c9-4fe1-8aeb-a44dce464681 service neutron] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:02.731038 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-c3103e73-90c9-4fe1-8aeb-a44dce464681 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:02.809629 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd52ed-6480-4f3e-bc98-42a2ae660b31 service neutron] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:02.809629 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-9efd52ed-6480-4f3e-bc98-42a2ae660b31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:02.820052 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3103e73-90c9-4fe1-8aeb-a44dce464681 service neutron] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:02.822670 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd52ed-6480-4f3e-bc98-42a2ae660b31 service neutron] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:02.826377 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607a3fa4-0e4e-47f0-bdb0-75d1879ff61c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '301d30bbd3de4cc69e96bb12729442c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:02.826982 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a3fa4-0e4e-47f0-bdb0-75d1879ff61c service placement] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:02.827258 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a3fa4-0e4e-47f0-bdb0-75d1879ff61c service placement] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:02.827530 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a3fa4-0e4e-47f0-bdb0-75d1879ff61c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:02.827655 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7779138-2721-4fa4-811d-e40aa43be337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:02.828371 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7779138-2721-4fa4-811d-e40aa43be337 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:02.828701 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7779138-2721-4fa4-811d-e40aa43be337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:02.828982 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7779138-2721-4fa4-811d-e40aa43be337 admin admin] PATH_INFO: `/v3/users/3246a8dea80d4d5b8c3fb50d5f1f31c0` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:02.829665 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-607a3fa4-0e4e-47f0-bdb0-75d1879ff61c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:02.830103 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7779138-2721-4fa4-811d-e40aa43be337 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3246a8dea80d4d5b8c3fb50d5f1f31c0)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:02.832244 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7779138-2721-4fa4-811d-e40aa43be337 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:02.842127 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607a3fa4-0e4e-47f0-bdb0-75d1879ff61c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:02.843727 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607a3fa4-0e4e-47f0-bdb0-75d1879ff61c service placement] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:02.854890 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1255/2499] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:21:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:21:02.885141 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-b7779138-2721-4fa4-811d-e40aa43be337 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3246a8dea80d4d5b8c3fb50d5f1f31c0'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:02.897653 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-b7779138-2721-4fa4-811d-e40aa43be337 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3246a8dea80d4d5b8c3fb50d5f1f31c0'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:02.905313 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-b7779138-2721-4fa4-811d-e40aa43be337 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3246a8dea80d4d5b8c3fb50d5f1f31c0'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:02.906026 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-b7779138-2721-4fa4-811d-e40aa43be337 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3246a8dea80d4d5b8c3fb50d5f1f31c0'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:02.913583 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-b7779138-2721-4fa4-811d-e40aa43be337 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3246a8dea80d4d5b8c3fb50d5f1f31c0'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:02.920121 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1245/2500] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:21:02 2022] DELETE /identity/v3/users/3246a8dea80d4d5b8c3fb50d5f1f31c0 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:21:03.092611 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607a3fa4-0e4e-47f0-bdb0-75d1879ff61c service placement] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:03.103897 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607a3fa4-0e4e-47f0-bdb0-75d1879ff61c service placement] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:03.162039 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ee8340-97d1-41e6-9311-cfee5d599002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:03.162666 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee8340-97d1-41e6-9311-cfee5d599002 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:03.162943 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee8340-97d1-41e6-9311-cfee5d599002 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:03.163218 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee8340-97d1-41e6-9311-cfee5d599002 admin admin] PATH_INFO: `/v3/projects/8c24dd087c274b6094aa720d81b11e98` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:03.164354 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ee8340-97d1-41e6-9311-cfee5d599002 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c24dd087c274b6094aa720d81b11e98)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:03.166956 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ee8340-97d1-41e6-9311-cfee5d599002 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:03.213360 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-23ee8340-97d1-41e6-9311-cfee5d599002 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c24dd087c274b6094aa720d81b11e98', 'request_id': 'req-23ee8340-97d1-41e6-9311-cfee5d599002'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:03.219452 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-23ee8340-97d1-41e6-9311-cfee5d599002 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c24dd087c274b6094aa720d81b11e98', 'request_id': 'req-23ee8340-97d1-41e6-9311-cfee5d599002'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:03.224945 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-23ee8340-97d1-41e6-9311-cfee5d599002 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c24dd087c274b6094aa720d81b11e98', 'request_id': 'req-23ee8340-97d1-41e6-9311-cfee5d599002'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:03.238156 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-23ee8340-97d1-41e6-9311-cfee5d599002 admin admin] The token cache is being invalidate because project 8c24dd087c274b6094aa720d81b11e98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:21:03.238451 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-23ee8340-97d1-41e6-9311-cfee5d599002 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:03.239105 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-23ee8340-97d1-41e6-9311-cfee5d599002 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:03.240659 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1256/2501] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:21:03 2022] DELETE /identity/v3/projects/8c24dd087c274b6094aa720d81b11e98 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:21:03.245832 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500da70d-10f9-4b46-90ae-0e097b139a64 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:03.246136 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500da70d-10f9-4b46-90ae-0e097b139a64 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:03.246405 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500da70d-10f9-4b46-90ae-0e097b139a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:03.266023 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-500da70d-10f9-4b46-90ae-0e097b139a64 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:21:03.267244 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-500da70d-10f9-4b46-90ae-0e097b139a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:03.326248 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1246/2502] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:21:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:21:03.334095 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645f45af-2608-454d-a8d0-c515815b7115 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:03.334385 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645f45af-2608-454d-a8d0-c515815b7115 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:03.334665 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645f45af-2608-454d-a8d0-c515815b7115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:03.355307 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-645f45af-2608-454d-a8d0-c515815b7115 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:21:03.356566 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-645f45af-2608-454d-a8d0-c515815b7115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:03.364455 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1257/2503] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:21:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:21:03.368572 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500da70d-10f9-4b46-90ae-0e097b139a64 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:03.378238 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500da70d-10f9-4b46-90ae-0e097b139a64 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:03.382707 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a09997-2ac0-405b-983b-553ee845ec66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:03.383550 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a09997-2ac0-405b-983b-553ee845ec66 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:03.383836 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a09997-2ac0-405b-983b-553ee845ec66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:03.384106 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a09997-2ac0-405b-983b-553ee845ec66 admin admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:03.384895 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a09997-2ac0-405b-983b-553ee845ec66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:21:03.385186 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a09997-2ac0-405b-983b-553ee845ec66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:03.386783 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a09997-2ac0-405b-983b-553ee845ec66 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:03.388232 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:21:03.399730 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1247/2504] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:21:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:21:03.404429 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645f45af-2608-454d-a8d0-c515815b7115 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:03.413595 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645f45af-2608-454d-a8d0-c515815b7115 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:03.419127 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b445621-df90-4647-aba2-4f773f8f4534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:03.419709 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b445621-df90-4647-aba2-4f773f8f4534 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:03.419984 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b445621-df90-4647-aba2-4f773f8f4534 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:03.420284 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b445621-df90-4647-aba2-4f773f8f4534 admin admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:03.420902 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b445621-df90-4647-aba2-4f773f8f4534 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:03.422790 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b445621-df90-4647-aba2-4f773f8f4534 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:03.434029 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:21:03.450983 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1258/2505] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:21:03 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:21:03.455029 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a09997-2ac0-405b-983b-553ee845ec66 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:03.465492 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a09997-2ac0-405b-983b-553ee845ec66 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:03.522054 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c9f54-1a80-4fce-9b8a-c5635ef4234b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:03.522652 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c9f54-1a80-4fce-9b8a-c5635ef4234b admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:03.522935 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c9f54-1a80-4fce-9b8a-c5635ef4234b admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:03.523209 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c9f54-1a80-4fce-9b8a-c5635ef4234b admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:03.523830 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005c9f54-1a80-4fce-9b8a-c5635ef4234b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:03.526640 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005c9f54-1a80-4fce-9b8a-c5635ef4234b admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:03.547960 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1248/2506] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:21:03 2022] POST /identity/v3/users => generated 286 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:21:03.552231 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b445621-df90-4647-aba2-4f773f8f4534 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:03.563552 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b445621-df90-4647-aba2-4f773f8f4534 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:03.568318 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ada4e69-15b5-4b7d-bd20-b796799730ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:03.568976 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada4e69-15b5-4b7d-bd20-b796799730ad admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:03.569278 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada4e69-15b5-4b7d-bd20-b796799730ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:03.569552 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada4e69-15b5-4b7d-bd20-b796799730ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:03.570218 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ada4e69-15b5-4b7d-bd20-b796799730ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:03.572384 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ada4e69-15b5-4b7d-bd20-b796799730ad admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:03.582396 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1259/2507] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:21:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:21:03.586561 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c9f54-1a80-4fce-9b8a-c5635ef4234b admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:03.596912 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c9f54-1a80-4fce-9b8a-c5635ef4234b admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:03.601631 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a435c2-a196-403c-a7c4-0d7c79f4333b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:03.602259 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a435c2-a196-403c-a7c4-0d7c79f4333b admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:03.602541 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a435c2-a196-403c-a7c4-0d7c79f4333b admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:03.602843 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a435c2-a196-403c-a7c4-0d7c79f4333b admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:03.603732 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a435c2-a196-403c-a7c4-0d7c79f4333b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:03.605438 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a435c2-a196-403c-a7c4-0d7c79f4333b admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:03.615967 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1249/2508] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:21:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:21:03.620283 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ada4e69-15b5-4b7d-bd20-b796799730ad admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:03.630364 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ada4e69-15b5-4b7d-bd20-b796799730ad admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:03.636199 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad28651-37bd-4f5f-bed7-98c3a8190129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:03.636773 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad28651-37bd-4f5f-bed7-98c3a8190129 admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:03.637130 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad28651-37bd-4f5f-bed7-98c3a8190129 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:03.637443 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad28651-37bd-4f5f-bed7-98c3a8190129 admin admin] PATH_INFO: `/v3/projects/2fb8a6edc25f4d21859b212f7a154771/users/b8be243515cb4aae95e8c24d768e0071/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:03.653307 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad28651-37bd-4f5f-bed7-98c3a8190129 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fb8a6edc25f4d21859b212f7a154771, user_id=b8be243515cb4aae95e8c24d768e0071, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:03.655959 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad28651-37bd-4f5f-bed7-98c3a8190129 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:03.668652 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1260/2509] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:21:03 2022] PUT /identity/v3/projects/2fb8a6edc25f4d21859b212f7a154771/users/b8be243515cb4aae95e8c24d768e0071/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:21:03.678741 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ae051-82e2-4ca3-9680-86c0805fcbc2 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:03.679015 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ae051-82e2-4ca3-9680-86c0805fcbc2 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:03.679293 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ae051-82e2-4ca3-9680-86c0805fcbc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:03.707034 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-dd9ae051-82e2-4ca3-9680-86c0805fcbc2 None None] MFA Rules not processed for user `b8be243515cb4aae95e8c24d768e0071`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:21:03.708279 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-dd9ae051-82e2-4ca3-9680-86c0805fcbc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:03.771871 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1250/2510] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:21:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:21:03.784742 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad28651-37bd-4f5f-bed7-98c3a8190129 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:03.786418 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-5ad28651-37bd-4f5f-bed7-98c3a8190129 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:03.860344 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad28651-37bd-4f5f-bed7-98c3a8190129 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:03.865934 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a76a17-eb38-4ad6-9e66-554d8afd7c6a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae643cfdcb164aa9b474747cd21aa116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:03.866563 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a76a17-eb38-4ad6-9e66-554d8afd7c6a service nova] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:03.866845 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a76a17-eb38-4ad6-9e66-554d8afd7c6a service nova] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:03.867117 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a76a17-eb38-4ad6-9e66-554d8afd7c6a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:03.875097 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a76a17-eb38-4ad6-9e66-554d8afd7c6a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:03.876859 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a76a17-eb38-4ad6-9e66-554d8afd7c6a service nova] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:03.886489 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1261/2511] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:21:03 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:21:06.582014 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9ae051-82e2-4ca3-9680-86c0805fcbc2 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:06.642434 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9ae051-82e2-4ca3-9680-86c0805fcbc2 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:06.646988 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4322aad-5199-486b-83f1-1a28106abb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:06.647613 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4322aad-5199-486b-83f1-1a28106abb81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:06.647894 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4322aad-5199-486b-83f1-1a28106abb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:06.648199 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4322aad-5199-486b-83f1-1a28106abb81 admin admin] PATH_INFO: `/v3/users/b8be243515cb4aae95e8c24d768e0071` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:06.649311 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4322aad-5199-486b-83f1-1a28106abb81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8be243515cb4aae95e8c24d768e0071)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:06.651893 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4322aad-5199-486b-83f1-1a28106abb81 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:06.654161 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:21:06.698980 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-f4322aad-5199-486b-83f1-1a28106abb81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8be243515cb4aae95e8c24d768e0071'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:06.710730 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-f4322aad-5199-486b-83f1-1a28106abb81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8be243515cb4aae95e8c24d768e0071'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:06.718353 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-f4322aad-5199-486b-83f1-1a28106abb81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8be243515cb4aae95e8c24d768e0071'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:06.719020 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-f4322aad-5199-486b-83f1-1a28106abb81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8be243515cb4aae95e8c24d768e0071'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:06.725836 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-f4322aad-5199-486b-83f1-1a28106abb81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8be243515cb4aae95e8c24d768e0071'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:06.732158 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1251/2512] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:21:06 2022] DELETE /identity/v3/users/b8be243515cb4aae95e8c24d768e0071 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:21:06.736765 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f80e570-469f-40d2-b4d0-32eeceb62b0d None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:06.737276 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f80e570-469f-40d2-b4d0-32eeceb62b0d None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:06.737386 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f80e570-469f-40d2-b4d0-32eeceb62b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:06.756298 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-6f80e570-469f-40d2-b4d0-32eeceb62b0d None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:21:06.757489 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-6f80e570-469f-40d2-b4d0-32eeceb62b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:06.815947 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1262/2513] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:21:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:21:06.826209 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4322aad-5199-486b-83f1-1a28106abb81 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:06.827637 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-f4322aad-5199-486b-83f1-1a28106abb81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:06.883839 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4322aad-5199-486b-83f1-1a28106abb81 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:06.888199 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f9f39-4e08-4060-9bfb-75ee61fcac05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:06.888804 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f9f39-4e08-4060-9bfb-75ee61fcac05 service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:06.889082 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f9f39-4e08-4060-9bfb-75ee61fcac05 service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:06.889353 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f9f39-4e08-4060-9bfb-75ee61fcac05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:06.898347 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6f9f39-4e08-4060-9bfb-75ee61fcac05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:06.899792 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6f9f39-4e08-4060-9bfb-75ee61fcac05 service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:06.909899 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1252/2514] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:21:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:21:07.173382 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f80e570-469f-40d2-b4d0-32eeceb62b0d None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:07.174983 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-6f80e570-469f-40d2-b4d0-32eeceb62b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:07.188300 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f80e570-469f-40d2-b4d0-32eeceb62b0d None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:07.193473 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde141b0-b823-42ab-a435-d606bc9a86fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:07.194148 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde141b0-b823-42ab-a435-d606bc9a86fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:07.194431 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde141b0-b823-42ab-a435-d606bc9a86fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:07.194709 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde141b0-b823-42ab-a435-d606bc9a86fb admin admin] PATH_INFO: `/v3/projects/2fb8a6edc25f4d21859b212f7a154771` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:07.195869 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde141b0-b823-42ab-a435-d606bc9a86fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fb8a6edc25f4d21859b212f7a154771)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:07.198606 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde141b0-b823-42ab-a435-d606bc9a86fb admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:07.237160 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:21:07.248705 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-bde141b0-b823-42ab-a435-d606bc9a86fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fb8a6edc25f4d21859b212f7a154771', 'request_id': 'req-bde141b0-b823-42ab-a435-d606bc9a86fb'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:07.254990 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-bde141b0-b823-42ab-a435-d606bc9a86fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fb8a6edc25f4d21859b212f7a154771', 'request_id': 'req-bde141b0-b823-42ab-a435-d606bc9a86fb'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:07.261867 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-bde141b0-b823-42ab-a435-d606bc9a86fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fb8a6edc25f4d21859b212f7a154771', 'request_id': 'req-bde141b0-b823-42ab-a435-d606bc9a86fb'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:07.273848 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-bde141b0-b823-42ab-a435-d606bc9a86fb admin admin] The token cache is being invalidate because project 2fb8a6edc25f4d21859b212f7a154771 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:21:07.274135 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-bde141b0-b823-42ab-a435-d606bc9a86fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:07.274755 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-bde141b0-b823-42ab-a435-d606bc9a86fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:07.276484 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1263/2515] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:21:07 2022] DELETE /identity/v3/projects/2fb8a6edc25f4d21859b212f7a154771 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:21:07.282334 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceba796-7d5c-4b98-a899-8382fc64140f None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:07.282653 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceba796-7d5c-4b98-a899-8382fc64140f None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:07.282938 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceba796-7d5c-4b98-a899-8382fc64140f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:07.304604 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-aceba796-7d5c-4b98-a899-8382fc64140f None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:21:07.305826 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-aceba796-7d5c-4b98-a899-8382fc64140f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:07.365365 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1253/2516] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:21:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:21:07.375196 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c04dbc9-6d4c-4196-81cb-1ebb42ec1559 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:07.375481 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c04dbc9-6d4c-4196-81cb-1ebb42ec1559 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:07.375757 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c04dbc9-6d4c-4196-81cb-1ebb42ec1559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:07.395484 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-2c04dbc9-6d4c-4196-81cb-1ebb42ec1559 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:21:07.396704 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-2c04dbc9-6d4c-4196-81cb-1ebb42ec1559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:07.404856 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1264/2517] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:21:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:21:07.409001 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceba796-7d5c-4b98-a899-8382fc64140f None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:07.419430 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceba796-7d5c-4b98-a899-8382fc64140f None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:07.424069 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac3b365-ff0a-4781-a005-62fd156016c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:07.424701 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3b365-ff0a-4781-a005-62fd156016c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:07.424984 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3b365-ff0a-4781-a005-62fd156016c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:07.425257 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3b365-ff0a-4781-a005-62fd156016c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:07.425904 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac3b365-ff0a-4781-a005-62fd156016c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:21:07.426207 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac3b365-ff0a-4781-a005-62fd156016c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:07.427825 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac3b365-ff0a-4781-a005-62fd156016c1 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:07.439800 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1254/2518] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:21:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:21:07.443904 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c04dbc9-6d4c-4196-81cb-1ebb42ec1559 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:07.453427 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c04dbc9-6d4c-4196-81cb-1ebb42ec1559 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:07.458176 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b7d0e2-2354-4594-b06f-70960b9ab86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:07.458794 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b7d0e2-2354-4594-b06f-70960b9ab86f admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:07.459070 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b7d0e2-2354-4594-b06f-70960b9ab86f admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:07.459336 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b7d0e2-2354-4594-b06f-70960b9ab86f admin admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:07.459966 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b7d0e2-2354-4594-b06f-70960b9ab86f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:07.461946 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b7d0e2-2354-4594-b06f-70960b9ab86f admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:07.489264 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1265/2519] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:21:07 2022] POST /identity/v3/projects => generated 393 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:21:07.493361 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac3b365-ff0a-4781-a005-62fd156016c1 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:07.503935 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac3b365-ff0a-4781-a005-62fd156016c1 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:07.512209 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:21:07.562717 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccd0bd3-ba49-4697-8385-e1a427a82afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:07.563317 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd0bd3-ba49-4697-8385-e1a427a82afe admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:07.563596 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd0bd3-ba49-4697-8385-e1a427a82afe admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:07.563867 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd0bd3-ba49-4697-8385-e1a427a82afe admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:07.564531 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccd0bd3-ba49-4697-8385-e1a427a82afe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:07.566468 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccd0bd3-ba49-4697-8385-e1a427a82afe admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:07.587078 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1255/2520] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:21:07 2022] POST /identity/v3/users => generated 297 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:21:07.591280 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b7d0e2-2354-4594-b06f-70960b9ab86f admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:07.601608 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b7d0e2-2354-4594-b06f-70960b9ab86f admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:07.607183 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acaffd5-c297-474b-87b2-03b18582cd1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:07.607777 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acaffd5-c297-474b-87b2-03b18582cd1c admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:07.608055 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acaffd5-c297-474b-87b2-03b18582cd1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:07.608354 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acaffd5-c297-474b-87b2-03b18582cd1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:07.608985 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acaffd5-c297-474b-87b2-03b18582cd1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:07.610624 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acaffd5-c297-474b-87b2-03b18582cd1c admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:07.621374 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1266/2521] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:21:07.625331 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccd0bd3-ba49-4697-8385-e1a427a82afe admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:07.635868 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccd0bd3-ba49-4697-8385-e1a427a82afe admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:07.640700 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb2567d-33f0-49f9-861b-d86bdfe322fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:07.641296 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb2567d-33f0-49f9-861b-d86bdfe322fd admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:07.641607 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb2567d-33f0-49f9-861b-d86bdfe322fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:07.641926 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb2567d-33f0-49f9-861b-d86bdfe322fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:07.642766 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb2567d-33f0-49f9-861b-d86bdfe322fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:07.644433 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb2567d-33f0-49f9-861b-d86bdfe322fd admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:07.656208 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1256/2522] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:21:07.660302 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acaffd5-c297-474b-87b2-03b18582cd1c admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:07.670815 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acaffd5-c297-474b-87b2-03b18582cd1c admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:07.675330 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6574f166-f5f7-457d-8fb3-902c5a34f3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:07.675966 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574f166-f5f7-457d-8fb3-902c5a34f3f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:07.676282 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574f166-f5f7-457d-8fb3-902c5a34f3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:07.676554 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574f166-f5f7-457d-8fb3-902c5a34f3f9 admin admin] PATH_INFO: `/v3/projects/ce26960c7a4943b9a38c6aeb8f133895/users/079ab2a591bc47e4bf52c54debe90bd9/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:07.691316 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6574f166-f5f7-457d-8fb3-902c5a34f3f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce26960c7a4943b9a38c6aeb8f133895, user_id=079ab2a591bc47e4bf52c54debe90bd9, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:07.694015 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6574f166-f5f7-457d-8fb3-902c5a34f3f9 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:07.706917 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1267/2523] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:21:07 2022] PUT /identity/v3/projects/ce26960c7a4943b9a38c6aeb8f133895/users/079ab2a591bc47e4bf52c54debe90bd9/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:21:07.718135 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab94727-4244-4d4c-93b8-fe2611e035e2 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:07.718418 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab94727-4244-4d4c-93b8-fe2611e035e2 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:07.718692 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab94727-4244-4d4c-93b8-fe2611e035e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:07.746778 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-9ab94727-4244-4d4c-93b8-fe2611e035e2 None None] MFA Rules not processed for user `079ab2a591bc47e4bf52c54debe90bd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:21:07.747997 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-9ab94727-4244-4d4c-93b8-fe2611e035e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:07.810951 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1257/2524] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:21:07 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:21:07.815480 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6574f166-f5f7-457d-8fb3-902c5a34f3f9 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:07.877748 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6574f166-f5f7-457d-8fb3-902c5a34f3f9 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:07.882291 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16785d4f-d086-4b61-8f05-cbdbe2f4600c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:07.882883 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16785d4f-d086-4b61-8f05-cbdbe2f4600c admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:07.883158 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16785d4f-d086-4b61-8f05-cbdbe2f4600c admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:07.883695 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16785d4f-d086-4b61-8f05-cbdbe2f4600c admin admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:07.884410 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16785d4f-d086-4b61-8f05-cbdbe2f4600c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:07.886503 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16785d4f-d086-4b61-8f05-cbdbe2f4600c admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:07.913007 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1268/2525] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:21:07 2022] POST /identity/v3/projects => generated 393 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:21:07.917819 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab94727-4244-4d4c-93b8-fe2611e035e2 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:07.928257 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab94727-4244-4d4c-93b8-fe2611e035e2 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:07.984572 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1356370e-1616-4775-91bf-fcaec603fa07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:07.985172 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1356370e-1616-4775-91bf-fcaec603fa07 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:07.985450 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1356370e-1616-4775-91bf-fcaec603fa07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:07.985720 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1356370e-1616-4775-91bf-fcaec603fa07 admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:07.986345 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1356370e-1616-4775-91bf-fcaec603fa07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:07.988283 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1356370e-1616-4775-91bf-fcaec603fa07 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:08.010006 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1258/2526] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:21:07 2022] POST /identity/v3/users => generated 303 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:21:08.014184 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16785d4f-d086-4b61-8f05-cbdbe2f4600c admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:08.024500 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16785d4f-d086-4b61-8f05-cbdbe2f4600c admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:08.029764 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1009aa81-c37b-4adb-a116-1b8961d262e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:08.030360 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1009aa81-c37b-4adb-a116-1b8961d262e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:08.030642 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1009aa81-c37b-4adb-a116-1b8961d262e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:08.030917 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1009aa81-c37b-4adb-a116-1b8961d262e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:08.031558 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1009aa81-c37b-4adb-a116-1b8961d262e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:08.033195 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1009aa81-c37b-4adb-a116-1b8961d262e2 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:08.044776 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1269/2527] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:21:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:21:08.049121 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1356370e-1616-4775-91bf-fcaec603fa07 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:08.059058 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1356370e-1616-4775-91bf-fcaec603fa07 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:08.063727 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978dd64f-cff6-47b5-8f19-4b3a2af95cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:08.064405 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978dd64f-cff6-47b5-8f19-4b3a2af95cbc admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:08.064688 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978dd64f-cff6-47b5-8f19-4b3a2af95cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:08.064967 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978dd64f-cff6-47b5-8f19-4b3a2af95cbc admin admin] PATH_INFO: `/v3/projects/18b8f053aa824808b05191e33c34bc69/users/24f4aa34915c47a39ce403e2afa3cd96/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:08.068599 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:21:08.081432 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978dd64f-cff6-47b5-8f19-4b3a2af95cbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=18b8f053aa824808b05191e33c34bc69, user_id=24f4aa34915c47a39ce403e2afa3cd96, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:08.084102 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978dd64f-cff6-47b5-8f19-4b3a2af95cbc admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:08.096966 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1259/2528] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:21:08 2022] PUT /identity/v3/projects/18b8f053aa824808b05191e33c34bc69/users/24f4aa34915c47a39ce403e2afa3cd96/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:21:08.100954 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1009aa81-c37b-4adb-a116-1b8961d262e2 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:08.110904 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1009aa81-c37b-4adb-a116-1b8961d262e2 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:08.143705 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:21:08.170032 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19203375-e5ce-4e71-9f73-eac4ad4d4930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:08.170633 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19203375-e5ce-4e71-9f73-eac4ad4d4930 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:08.170909 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19203375-e5ce-4e71-9f73-eac4ad4d4930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:08.171175 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19203375-e5ce-4e71-9f73-eac4ad4d4930 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:08.171807 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19203375-e5ce-4e71-9f73-eac4ad4d4930 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:08.173444 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19203375-e5ce-4e71-9f73-eac4ad4d4930 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:08.183396 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1270/2529] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:21:08 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:21:08.187697 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978dd64f-cff6-47b5-8f19-4b3a2af95cbc admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:08.198611 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978dd64f-cff6-47b5-8f19-4b3a2af95cbc admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:08.203207 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e81b83-1743-4343-b942-d2da6cb64055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:08.203847 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e81b83-1743-4343-b942-d2da6cb64055 admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:08.204124 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e81b83-1743-4343-b942-d2da6cb64055 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:08.204419 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e81b83-1743-4343-b942-d2da6cb64055 admin admin] PATH_INFO: `/v3/projects/18b8f053aa824808b05191e33c34bc69/users/24f4aa34915c47a39ce403e2afa3cd96/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:08.206303 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e81b83-1743-4343-b942-d2da6cb64055 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18b8f053aa824808b05191e33c34bc69, user_id=24f4aa34915c47a39ce403e2afa3cd96, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:08.208983 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e81b83-1743-4343-b942-d2da6cb64055 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:08.219652 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1260/2530] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:21:08 2022] PUT /identity/v3/projects/18b8f053aa824808b05191e33c34bc69/users/24f4aa34915c47a39ce403e2afa3cd96/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:21:08.230051 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f0c444-80e2-4c6c-bb62-8f4c154d930c None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:08.230336 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f0c444-80e2-4c6c-bb62-8f4c154d930c None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:08.230611 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f0c444-80e2-4c6c-bb62-8f4c154d930c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:08.258750 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-b8f0c444-80e2-4c6c-bb62-8f4c154d930c None None] MFA Rules not processed for user `24f4aa34915c47a39ce403e2afa3cd96`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:21:08.260574 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-b8f0c444-80e2-4c6c-bb62-8f4c154d930c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:08.332021 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1271/2531] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:21:08 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:21:08.343650 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e81b83-1743-4343-b942-d2da6cb64055 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:08.345407 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-d2e81b83-1743-4343-b942-d2da6cb64055 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:08.406401 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e81b83-1743-4343-b942-d2da6cb64055 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:08.410093 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c39c25-3b33-4ae6-88b5-4637b75d1f4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:08.410692 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c39c25-3b33-4ae6-88b5-4637b75d1f4a service cinder] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:08.410970 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c39c25-3b33-4ae6-88b5-4637b75d1f4a service cinder] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:08.411240 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c39c25-3b33-4ae6-88b5-4637b75d1f4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:08.420514 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c39c25-3b33-4ae6-88b5-4637b75d1f4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:08.421943 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c39c25-3b33-4ae6-88b5-4637b75d1f4a service cinder] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:08.432000 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1261/2532] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:21:08 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:21:08.478223 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f0c444-80e2-4c6c-bb62-8f4c154d930c None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:08.479905 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-b8f0c444-80e2-4c6c-bb62-8f4c154d930c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:08.491814 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f0c444-80e2-4c6c-bb62-8f4c154d930c None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:08.496160 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b270cdb-236a-4685-bac4-47ce48b353f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:08.496762 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b270cdb-236a-4685-bac4-47ce48b353f4 service cinder] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:08.497040 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b270cdb-236a-4685-bac4-47ce48b353f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:08.497318 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b270cdb-236a-4685-bac4-47ce48b353f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:08.555780 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b270cdb-236a-4685-bac4-47ce48b353f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:08.557305 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b270cdb-236a-4685-bac4-47ce48b353f4 service cinder] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:08.566458 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1272/2533] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:21:08 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:21:12.004062 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c39c25-3b33-4ae6-88b5-4637b75d1f4a service cinder] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:12.068502 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c39c25-3b33-4ae6-88b5-4637b75d1f4a service cinder] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:12.073904 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a830b6cd-c3d8-40f6-bc82-ffcac6807575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:12.074523 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a830b6cd-c3d8-40f6-bc82-ffcac6807575 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:12.074802 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a830b6cd-c3d8-40f6-bc82-ffcac6807575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:12.075073 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a830b6cd-c3d8-40f6-bc82-ffcac6807575 admin admin] PATH_INFO: `/v3/users/079ab2a591bc47e4bf52c54debe90bd9` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:12.076203 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a830b6cd-c3d8-40f6-bc82-ffcac6807575 admin admin] RBAC: Authorizing `identity:delete_user(user_id=079ab2a591bc47e4bf52c54debe90bd9)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:12.078146 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a830b6cd-c3d8-40f6-bc82-ffcac6807575 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:12.123977 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-a830b6cd-c3d8-40f6-bc82-ffcac6807575 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '079ab2a591bc47e4bf52c54debe90bd9'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:12.136839 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-a830b6cd-c3d8-40f6-bc82-ffcac6807575 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '079ab2a591bc47e4bf52c54debe90bd9'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:12.143869 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-a830b6cd-c3d8-40f6-bc82-ffcac6807575 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '079ab2a591bc47e4bf52c54debe90bd9'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:12.144565 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-a830b6cd-c3d8-40f6-bc82-ffcac6807575 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '079ab2a591bc47e4bf52c54debe90bd9'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:12.147176 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:21:12.153618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-a830b6cd-c3d8-40f6-bc82-ffcac6807575 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '079ab2a591bc47e4bf52c54debe90bd9'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:12.160870 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1262/2534] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:21:12 2022] DELETE /identity/v3/users/079ab2a591bc47e4bf52c54debe90bd9 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:21:12.165569 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79136e9-6acc-488d-a797-e4c5df8212f0 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:12.166062 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79136e9-6acc-488d-a797-e4c5df8212f0 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:12.166170 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79136e9-6acc-488d-a797-e4c5df8212f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:12.185646 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-b79136e9-6acc-488d-a797-e4c5df8212f0 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:21:12.186853 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-b79136e9-6acc-488d-a797-e4c5df8212f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:12.234503 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:21:12.247128 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1273/2535] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:21:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:21:12.256989 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a830b6cd-c3d8-40f6-bc82-ffcac6807575 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:12.258695 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-a830b6cd-c3d8-40f6-bc82-ffcac6807575 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:12.318668 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a830b6cd-c3d8-40f6-bc82-ffcac6807575 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:12.322608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc26840-e59d-49ca-8ddf-9e4687bf6be3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:12.323229 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc26840-e59d-49ca-8ddf-9e4687bf6be3 service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:12.323506 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc26840-e59d-49ca-8ddf-9e4687bf6be3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:12.323774 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc26840-e59d-49ca-8ddf-9e4687bf6be3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:12.332673 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc26840-e59d-49ca-8ddf-9e4687bf6be3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:12.334090 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc26840-e59d-49ca-8ddf-9e4687bf6be3 service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:12.344211 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1263/2536] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:21:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:21:12.617589 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79136e9-6acc-488d-a797-e4c5df8212f0 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:12.619236 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-b79136e9-6acc-488d-a797-e4c5df8212f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:12.632749 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79136e9-6acc-488d-a797-e4c5df8212f0 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:12.640747 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe7e6a5-3792-4409-be71-d7cf41e85359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:12.641327 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe7e6a5-3792-4409-be71-d7cf41e85359 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:12.641643 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe7e6a5-3792-4409-be71-d7cf41e85359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:12.641928 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe7e6a5-3792-4409-be71-d7cf41e85359 admin admin] PATH_INFO: `/v3/projects/ce26960c7a4943b9a38c6aeb8f133895` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:12.643024 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe7e6a5-3792-4409-be71-d7cf41e85359 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce26960c7a4943b9a38c6aeb8f133895)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:12.645449 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe7e6a5-3792-4409-be71-d7cf41e85359 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:12.692869 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-dfe7e6a5-3792-4409-be71-d7cf41e85359 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce26960c7a4943b9a38c6aeb8f133895', 'request_id': 'req-dfe7e6a5-3792-4409-be71-d7cf41e85359'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:12.699544 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-dfe7e6a5-3792-4409-be71-d7cf41e85359 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce26960c7a4943b9a38c6aeb8f133895', 'request_id': 'req-dfe7e6a5-3792-4409-be71-d7cf41e85359'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:12.705238 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-dfe7e6a5-3792-4409-be71-d7cf41e85359 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce26960c7a4943b9a38c6aeb8f133895', 'request_id': 'req-dfe7e6a5-3792-4409-be71-d7cf41e85359'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:12.718502 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-dfe7e6a5-3792-4409-be71-d7cf41e85359 admin admin] The token cache is being invalidate because project ce26960c7a4943b9a38c6aeb8f133895 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:21:12.718793 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-dfe7e6a5-3792-4409-be71-d7cf41e85359 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:12.719408 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-dfe7e6a5-3792-4409-be71-d7cf41e85359 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:12.720924 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1274/2537] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:21:12 2022] DELETE /identity/v3/projects/ce26960c7a4943b9a38c6aeb8f133895 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:21:12.725062 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc26840-e59d-49ca-8ddf-9e4687bf6be3 service neutron] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:12.734820 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc26840-e59d-49ca-8ddf-9e4687bf6be3 service neutron] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:12.791523 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a739a-6e0c-4720-986b-9307f16ff63d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:12.792163 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a739a-6e0c-4720-986b-9307f16ff63d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:12.792453 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a739a-6e0c-4720-986b-9307f16ff63d admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:12.792724 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a739a-6e0c-4720-986b-9307f16ff63d admin admin] PATH_INFO: `/v3/users/24f4aa34915c47a39ce403e2afa3cd96` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:12.793829 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1a739a-6e0c-4720-986b-9307f16ff63d admin admin] RBAC: Authorizing `identity:delete_user(user_id=24f4aa34915c47a39ce403e2afa3cd96)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:12.795769 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1a739a-6e0c-4720-986b-9307f16ff63d admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:12.841214 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-ec1a739a-6e0c-4720-986b-9307f16ff63d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24f4aa34915c47a39ce403e2afa3cd96'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:12.853465 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-ec1a739a-6e0c-4720-986b-9307f16ff63d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24f4aa34915c47a39ce403e2afa3cd96'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:12.860421 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-ec1a739a-6e0c-4720-986b-9307f16ff63d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24f4aa34915c47a39ce403e2afa3cd96'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:12.861094 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-ec1a739a-6e0c-4720-986b-9307f16ff63d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24f4aa34915c47a39ce403e2afa3cd96'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:12.868653 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-ec1a739a-6e0c-4720-986b-9307f16ff63d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24f4aa34915c47a39ce403e2afa3cd96'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:12.874673 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1264/2538] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:21:12 2022] DELETE /identity/v3/users/24f4aa34915c47a39ce403e2afa3cd96 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:21:13.147558 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe7e6a5-3792-4409-be71-d7cf41e85359 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:13.149147 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-dfe7e6a5-3792-4409-be71-d7cf41e85359 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:13.163010 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe7e6a5-3792-4409-be71-d7cf41e85359 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:13.220599 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e6c064-ed27-4f37-9f2e-62616534194a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:13.221347 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e6c064-ed27-4f37-9f2e-62616534194a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:13.221496 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e6c064-ed27-4f37-9f2e-62616534194a admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:13.221771 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e6c064-ed27-4f37-9f2e-62616534194a admin admin] PATH_INFO: `/v3/projects/18b8f053aa824808b05191e33c34bc69` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:13.222931 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e6c064-ed27-4f37-9f2e-62616534194a admin admin] RBAC: Authorizing `identity:delete_project(project_id=18b8f053aa824808b05191e33c34bc69)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:13.224897 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e6c064-ed27-4f37-9f2e-62616534194a admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:13.270955 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-b4e6c064-ed27-4f37-9f2e-62616534194a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18b8f053aa824808b05191e33c34bc69', 'request_id': 'req-b4e6c064-ed27-4f37-9f2e-62616534194a'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:13.277271 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-b4e6c064-ed27-4f37-9f2e-62616534194a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18b8f053aa824808b05191e33c34bc69', 'request_id': 'req-b4e6c064-ed27-4f37-9f2e-62616534194a'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:13.282820 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-b4e6c064-ed27-4f37-9f2e-62616534194a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18b8f053aa824808b05191e33c34bc69', 'request_id': 'req-b4e6c064-ed27-4f37-9f2e-62616534194a'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:13.296967 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-b4e6c064-ed27-4f37-9f2e-62616534194a admin admin] The token cache is being invalidate because project 18b8f053aa824808b05191e33c34bc69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:21:13.297262 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-b4e6c064-ed27-4f37-9f2e-62616534194a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:13.297896 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-b4e6c064-ed27-4f37-9f2e-62616534194a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:13.299392 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1275/2539] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:21:13 2022] DELETE /identity/v3/projects/18b8f053aa824808b05191e33c34bc69 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:21:13.304397 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434393cd-b6e0-4d2b-b00b-b4632236a0ea None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:13.304860 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434393cd-b6e0-4d2b-b00b-b4632236a0ea None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:13.305006 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434393cd-b6e0-4d2b-b00b-b4632236a0ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:13.325440 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-434393cd-b6e0-4d2b-b00b-b4632236a0ea None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:21:13.326671 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-434393cd-b6e0-4d2b-b00b-b4632236a0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:13.335885 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:21:13.389870 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1265/2540] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:21:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:21:13.398840 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baec912-27ad-4c5b-9912-c79cfdf91e3b None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:13.399129 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baec912-27ad-4c5b-9912-c79cfdf91e3b None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:13.399407 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baec912-27ad-4c5b-9912-c79cfdf91e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:13.419453 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-9baec912-27ad-4c5b-9912-c79cfdf91e3b None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:21:13.420712 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-9baec912-27ad-4c5b-9912-c79cfdf91e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:13.428804 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1276/2541] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:21:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:21:13.433527 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434393cd-b6e0-4d2b-b00b-b4632236a0ea None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:13.443190 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434393cd-b6e0-4d2b-b00b-b4632236a0ea None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:13.447934 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4465fd33-2f3c-4c20-922d-38dabd35273d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:13.448563 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4465fd33-2f3c-4c20-922d-38dabd35273d admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:13.448846 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4465fd33-2f3c-4c20-922d-38dabd35273d admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:13.449121 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4465fd33-2f3c-4c20-922d-38dabd35273d admin admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:13.449765 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4465fd33-2f3c-4c20-922d-38dabd35273d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:21:13.450072 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4465fd33-2f3c-4c20-922d-38dabd35273d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:13.451694 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4465fd33-2f3c-4c20-922d-38dabd35273d admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:13.463170 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1266/2542] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:21:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:21:13.467255 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baec912-27ad-4c5b-9912-c79cfdf91e3b None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:13.471727 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:21:13.478603 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baec912-27ad-4c5b-9912-c79cfdf91e3b None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:13.485496 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b05e916-20f2-4710-9ddd-b3e764afc029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:13.486090 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05e916-20f2-4710-9ddd-b3e764afc029 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:13.486366 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05e916-20f2-4710-9ddd-b3e764afc029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:13.486634 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05e916-20f2-4710-9ddd-b3e764afc029 admin admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:13.487291 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b05e916-20f2-4710-9ddd-b3e764afc029 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:13.489223 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b05e916-20f2-4710-9ddd-b3e764afc029 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:13.516529 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1277/2543] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:21:13 2022] POST /identity/v3/projects => generated 407 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 01:21:13.520912 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4465fd33-2f3c-4c20-922d-38dabd35273d admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:13.532419 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4465fd33-2f3c-4c20-922d-38dabd35273d admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:13.592093 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74acd33b-c969-46a5-a7d8-f07fa33f123c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:13.592738 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74acd33b-c969-46a5-a7d8-f07fa33f123c admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:13.593021 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74acd33b-c969-46a5-a7d8-f07fa33f123c admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:13.593296 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74acd33b-c969-46a5-a7d8-f07fa33f123c admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:13.593938 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74acd33b-c969-46a5-a7d8-f07fa33f123c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:13.595870 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74acd33b-c969-46a5-a7d8-f07fa33f123c admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:13.620051 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1267/2544] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:21:13 2022] POST /identity/v3/users => generated 304 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:21:13.623993 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b05e916-20f2-4710-9ddd-b3e764afc029 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:13.635909 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b05e916-20f2-4710-9ddd-b3e764afc029 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:13.641428 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada2016e-29d8-4bbd-ae57-dbf0f7320b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:13.642152 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada2016e-29d8-4bbd-ae57-dbf0f7320b7d admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:13.642539 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada2016e-29d8-4bbd-ae57-dbf0f7320b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:13.642810 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada2016e-29d8-4bbd-ae57-dbf0f7320b7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:13.643474 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada2016e-29d8-4bbd-ae57-dbf0f7320b7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:13.645101 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada2016e-29d8-4bbd-ae57-dbf0f7320b7d admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:13.656542 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1278/2545] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:21:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 01:21:13.660568 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74acd33b-c969-46a5-a7d8-f07fa33f123c admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:13.671307 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74acd33b-c969-46a5-a7d8-f07fa33f123c admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:13.676164 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97c7bc0-b987-47bb-b7aa-51e4b424fa13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:13.676827 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c7bc0-b987-47bb-b7aa-51e4b424fa13 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:13.677195 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c7bc0-b987-47bb-b7aa-51e4b424fa13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:13.677528 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c7bc0-b987-47bb-b7aa-51e4b424fa13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:13.678261 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97c7bc0-b987-47bb-b7aa-51e4b424fa13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:13.679913 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97c7bc0-b987-47bb-b7aa-51e4b424fa13 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:13.691666 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1268/2546] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:21:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:21:13.695849 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada2016e-29d8-4bbd-ae57-dbf0f7320b7d admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:13.706417 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada2016e-29d8-4bbd-ae57-dbf0f7320b7d admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:13.711143 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c265b-1b0b-470c-b3fc-d0e4f2aea2d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:13.711803 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c265b-1b0b-470c-b3fc-d0e4f2aea2d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:13.712774 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c265b-1b0b-470c-b3fc-d0e4f2aea2d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:13.713214 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c265b-1b0b-470c-b3fc-d0e4f2aea2d8 admin admin] PATH_INFO: `/v3/projects/18e971552b3647288aec3346cd69ddc1/users/1bdf724fd86d47139eb22baf179f8eb3/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:13.730254 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0c265b-1b0b-470c-b3fc-d0e4f2aea2d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18e971552b3647288aec3346cd69ddc1, user_id=1bdf724fd86d47139eb22baf179f8eb3, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:13.732899 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0c265b-1b0b-470c-b3fc-d0e4f2aea2d8 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:13.746915 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1279/2547] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:21:13 2022] PUT /identity/v3/projects/18e971552b3647288aec3346cd69ddc1/users/1bdf724fd86d47139eb22baf179f8eb3/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:21:13.757853 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f038d438-5963-4602-843e-0b3462bedb6c None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:13.758226 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f038d438-5963-4602-843e-0b3462bedb6c None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:13.758833 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f038d438-5963-4602-843e-0b3462bedb6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:13.787592 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-f038d438-5963-4602-843e-0b3462bedb6c None None] MFA Rules not processed for user `1bdf724fd86d47139eb22baf179f8eb3`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:21:13.788962 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-f038d438-5963-4602-843e-0b3462bedb6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:13.864414 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1269/2548] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:21:13 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:21:13.869062 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c265b-1b0b-470c-b3fc-d0e4f2aea2d8 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:13.932498 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c265b-1b0b-470c-b3fc-d0e4f2aea2d8 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:13.937163 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bec482e-a578-4541-aa48-68f2cd7820d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:13.937764 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec482e-a578-4541-aa48-68f2cd7820d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:13.938047 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec482e-a578-4541-aa48-68f2cd7820d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:13.938315 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec482e-a578-4541-aa48-68f2cd7820d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:13.938980 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bec482e-a578-4541-aa48-68f2cd7820d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:13.940921 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bec482e-a578-4541-aa48-68f2cd7820d4 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:13.970876 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1280/2549] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:21:13 2022] POST /identity/v3/projects => generated 407 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:21:13.974873 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f038d438-5963-4602-843e-0b3462bedb6c None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:13.985997 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f038d438-5963-4602-843e-0b3462bedb6c None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:13.995020 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:21:14.045751 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db498d8-b89c-42f0-959b-4ba53968f00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:14.046348 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db498d8-b89c-42f0-959b-4ba53968f00d admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:14.046623 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db498d8-b89c-42f0-959b-4ba53968f00d admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:14.046890 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db498d8-b89c-42f0-959b-4ba53968f00d admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:14.047504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db498d8-b89c-42f0-959b-4ba53968f00d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:14.049446 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db498d8-b89c-42f0-959b-4ba53968f00d admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:14.071608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1270/2550] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:21:13 2022] POST /identity/v3/users => generated 310 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:21:14.075534 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bec482e-a578-4541-aa48-68f2cd7820d4 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:14.085753 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bec482e-a578-4541-aa48-68f2cd7820d4 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:14.090494 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d042227f-4665-4904-a64c-c0104defd705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:14.091078 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d042227f-4665-4904-a64c-c0104defd705 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:14.091363 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d042227f-4665-4904-a64c-c0104defd705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:14.091639 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d042227f-4665-4904-a64c-c0104defd705 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:14.092323 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d042227f-4665-4904-a64c-c0104defd705 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:14.093946 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d042227f-4665-4904-a64c-c0104defd705 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:14.104916 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1281/2551] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:21:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:21:14.108819 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db498d8-b89c-42f0-959b-4ba53968f00d admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:14.119667 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db498d8-b89c-42f0-959b-4ba53968f00d admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:14.124349 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27838ba6-db7e-4592-ad5f-8ddb22b0f6a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:14.124985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27838ba6-db7e-4592-ad5f-8ddb22b0f6a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:14.125260 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27838ba6-db7e-4592-ad5f-8ddb22b0f6a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:14.125528 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27838ba6-db7e-4592-ad5f-8ddb22b0f6a8 admin admin] PATH_INFO: `/v3/projects/24802648cdcf47f3918bd26d314755de/users/61f61b293d3e43e2a2bf07a7c50ceb60/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:14.140467 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27838ba6-db7e-4592-ad5f-8ddb22b0f6a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24802648cdcf47f3918bd26d314755de, user_id=61f61b293d3e43e2a2bf07a7c50ceb60, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:14.143129 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27838ba6-db7e-4592-ad5f-8ddb22b0f6a8 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:14.156098 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1271/2552] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:21:14 2022] PUT /identity/v3/projects/24802648cdcf47f3918bd26d314755de/users/61f61b293d3e43e2a2bf07a7c50ceb60/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:21:14.160016 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d042227f-4665-4904-a64c-c0104defd705 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:14.170495 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d042227f-4665-4904-a64c-c0104defd705 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:14.230388 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43710d8-6403-4f80-977d-298296df1a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:14.231158 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43710d8-6403-4f80-977d-298296df1a52 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:14.231447 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43710d8-6403-4f80-977d-298296df1a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:14.231718 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43710d8-6403-4f80-977d-298296df1a52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:14.232444 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43710d8-6403-4f80-977d-298296df1a52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:14.233976 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43710d8-6403-4f80-977d-298296df1a52 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:14.244499 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1282/2553] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:21:14 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:21:14.248959 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27838ba6-db7e-4592-ad5f-8ddb22b0f6a8 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:14.259572 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27838ba6-db7e-4592-ad5f-8ddb22b0f6a8 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:14.264192 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e169ef4a-0b9e-4c2c-b150-3e68040beded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:14.264844 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169ef4a-0b9e-4c2c-b150-3e68040beded admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:14.265121 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169ef4a-0b9e-4c2c-b150-3e68040beded admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:14.265393 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169ef4a-0b9e-4c2c-b150-3e68040beded admin admin] PATH_INFO: `/v3/projects/24802648cdcf47f3918bd26d314755de/users/61f61b293d3e43e2a2bf07a7c50ceb60/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:14.267307 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e169ef4a-0b9e-4c2c-b150-3e68040beded admin admin] RBAC: Authorizing `identity:create_grant(project_id=24802648cdcf47f3918bd26d314755de, user_id=61f61b293d3e43e2a2bf07a7c50ceb60, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:14.270029 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e169ef4a-0b9e-4c2c-b150-3e68040beded admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:14.280150 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1272/2554] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:21:14 2022] PUT /identity/v3/projects/24802648cdcf47f3918bd26d314755de/users/61f61b293d3e43e2a2bf07a7c50ceb60/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:21:14.292603 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978af2dc-ee64-44ec-ad87-88f8aefec44a None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:14.292888 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978af2dc-ee64-44ec-ad87-88f8aefec44a None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:14.293165 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978af2dc-ee64-44ec-ad87-88f8aefec44a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:14.321599 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-978af2dc-ee64-44ec-ad87-88f8aefec44a None None] MFA Rules not processed for user `61f61b293d3e43e2a2bf07a7c50ceb60`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:21:14.322945 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-978af2dc-ee64-44ec-ad87-88f8aefec44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:14.326988 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:21:14.392928 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1283/2555] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:21:14 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:21:14.404069 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e169ef4a-0b9e-4c2c-b150-3e68040beded admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:14.405834 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-e169ef4a-0b9e-4c2c-b150-3e68040beded admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:14.464388 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e169ef4a-0b9e-4c2c-b150-3e68040beded admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:14.468048 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6356b0-9bf6-44ee-ad0d-4014ac8b8eeb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:14.469303 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6356b0-9bf6-44ee-ad0d-4014ac8b8eeb service cinder] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:14.469580 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6356b0-9bf6-44ee-ad0d-4014ac8b8eeb service cinder] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:14.469848 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6356b0-9bf6-44ee-ad0d-4014ac8b8eeb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:14.479654 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6356b0-9bf6-44ee-ad0d-4014ac8b8eeb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:14.481108 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6356b0-9bf6-44ee-ad0d-4014ac8b8eeb service cinder] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:14.490743 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1273/2556] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:21:14 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:21:14.642913 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978af2dc-ee64-44ec-ad87-88f8aefec44a None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:14.653889 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978af2dc-ee64-44ec-ad87-88f8aefec44a None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:14.657517 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9eae3e-5b03-4e23-a31f-fb381afb9cb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:14.658104 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9eae3e-5b03-4e23-a31f-fb381afb9cb5 service cinder] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:14.658380 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9eae3e-5b03-4e23-a31f-fb381afb9cb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:14.658647 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9eae3e-5b03-4e23-a31f-fb381afb9cb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:14.715174 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9eae3e-5b03-4e23-a31f-fb381afb9cb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:14.716663 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9eae3e-5b03-4e23-a31f-fb381afb9cb5 service cinder] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:14.725500 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1284/2557] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:21:14 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:21:23.106035 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6356b0-9bf6-44ee-ad0d-4014ac8b8eeb service cinder] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:23.171620 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:21:23.214218 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6356b0-9bf6-44ee-ad0d-4014ac8b8eeb service cinder] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:23.218997 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7171fdb6-a25a-44f2-9aa9-eb5e63ee91ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:23.219639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7171fdb6-a25a-44f2-9aa9-eb5e63ee91ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:23.219921 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7171fdb6-a25a-44f2-9aa9-eb5e63ee91ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:23.220239 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7171fdb6-a25a-44f2-9aa9-eb5e63ee91ae admin admin] PATH_INFO: `/v3/users/1bdf724fd86d47139eb22baf179f8eb3` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:23.221394 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7171fdb6-a25a-44f2-9aa9-eb5e63ee91ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bdf724fd86d47139eb22baf179f8eb3)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:23.223830 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7171fdb6-a25a-44f2-9aa9-eb5e63ee91ae admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:23.274361 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-7171fdb6-a25a-44f2-9aa9-eb5e63ee91ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bdf724fd86d47139eb22baf179f8eb3'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:23.287389 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-7171fdb6-a25a-44f2-9aa9-eb5e63ee91ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bdf724fd86d47139eb22baf179f8eb3'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:23.295222 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-7171fdb6-a25a-44f2-9aa9-eb5e63ee91ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bdf724fd86d47139eb22baf179f8eb3'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:23.295928 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-7171fdb6-a25a-44f2-9aa9-eb5e63ee91ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bdf724fd86d47139eb22baf179f8eb3'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:23.303082 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-7171fdb6-a25a-44f2-9aa9-eb5e63ee91ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bdf724fd86d47139eb22baf179f8eb3'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:23.310016 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1274/2558] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:21:23 2022] DELETE /identity/v3/users/1bdf724fd86d47139eb22baf179f8eb3 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:21:23.314559 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662bc2e9-95f3-45c0-98ac-a35fd96acd97 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:23.315045 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662bc2e9-95f3-45c0-98ac-a35fd96acd97 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:23.315146 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662bc2e9-95f3-45c0-98ac-a35fd96acd97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:23.364643 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-662bc2e9-95f3-45c0-98ac-a35fd96acd97 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:21:23.365870 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-662bc2e9-95f3-45c0-98ac-a35fd96acd97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:23.424703 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1285/2559] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:21:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:21:23.434584 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7171fdb6-a25a-44f2-9aa9-eb5e63ee91ae admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:23.435994 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-7171fdb6-a25a-44f2-9aa9-eb5e63ee91ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:23.507737 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7171fdb6-a25a-44f2-9aa9-eb5e63ee91ae admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:23.511743 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf615e-f09a-4b51-b474-c98a354c9eaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:23.512370 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf615e-f09a-4b51-b474-c98a354c9eaf service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:23.512651 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf615e-f09a-4b51-b474-c98a354c9eaf service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:23.512922 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf615e-f09a-4b51-b474-c98a354c9eaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:23.521674 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbf615e-f09a-4b51-b474-c98a354c9eaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:23.523108 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbf615e-f09a-4b51-b474-c98a354c9eaf service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:23.534604 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1275/2560] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:21:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:21:23.799699 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662bc2e9-95f3-45c0-98ac-a35fd96acd97 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:23.801139 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-662bc2e9-95f3-45c0-98ac-a35fd96acd97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:23.813308 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662bc2e9-95f3-45c0-98ac-a35fd96acd97 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:23.817916 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16609d5-9d59-4892-b954-2365ebcc0651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:23.818550 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16609d5-9d59-4892-b954-2365ebcc0651 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:23.818967 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16609d5-9d59-4892-b954-2365ebcc0651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:23.819248 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16609d5-9d59-4892-b954-2365ebcc0651 admin admin] PATH_INFO: `/v3/projects/18e971552b3647288aec3346cd69ddc1` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:23.820422 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16609d5-9d59-4892-b954-2365ebcc0651 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18e971552b3647288aec3346cd69ddc1)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:23.822937 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16609d5-9d59-4892-b954-2365ebcc0651 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:23.831354 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:21:23.871431 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-c16609d5-9d59-4892-b954-2365ebcc0651 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18e971552b3647288aec3346cd69ddc1', 'request_id': 'req-c16609d5-9d59-4892-b954-2365ebcc0651'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:23.877535 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-c16609d5-9d59-4892-b954-2365ebcc0651 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18e971552b3647288aec3346cd69ddc1', 'request_id': 'req-c16609d5-9d59-4892-b954-2365ebcc0651'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:23.883120 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-c16609d5-9d59-4892-b954-2365ebcc0651 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18e971552b3647288aec3346cd69ddc1', 'request_id': 'req-c16609d5-9d59-4892-b954-2365ebcc0651'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:23.896127 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-c16609d5-9d59-4892-b954-2365ebcc0651 admin admin] The token cache is being invalidate because project 18e971552b3647288aec3346cd69ddc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:21:23.896445 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-c16609d5-9d59-4892-b954-2365ebcc0651 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:23.897069 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-c16609d5-9d59-4892-b954-2365ebcc0651 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:23.898543 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1286/2561] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:21:23 2022] DELETE /identity/v3/projects/18e971552b3647288aec3346cd69ddc1 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:21:23.903360 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf615e-f09a-4b51-b474-c98a354c9eaf service neutron] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:23.913046 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf615e-f09a-4b51-b474-c98a354c9eaf service neutron] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:23.941596 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:21:23.975204 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c4e823-9a21-4b7f-9860-a686e4b2c580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:23.975843 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c4e823-9a21-4b7f-9860-a686e4b2c580 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:23.976122 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c4e823-9a21-4b7f-9860-a686e4b2c580 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:23.976464 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c4e823-9a21-4b7f-9860-a686e4b2c580 admin admin] PATH_INFO: `/v3/users/61f61b293d3e43e2a2bf07a7c50ceb60` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:23.977989 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c4e823-9a21-4b7f-9860-a686e4b2c580 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61f61b293d3e43e2a2bf07a7c50ceb60)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:23.980211 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c4e823-9a21-4b7f-9860-a686e4b2c580 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:24.031364 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-f2c4e823-9a21-4b7f-9860-a686e4b2c580 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61f61b293d3e43e2a2bf07a7c50ceb60'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:24.044856 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-f2c4e823-9a21-4b7f-9860-a686e4b2c580 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61f61b293d3e43e2a2bf07a7c50ceb60'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:24.051659 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-f2c4e823-9a21-4b7f-9860-a686e4b2c580 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61f61b293d3e43e2a2bf07a7c50ceb60'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:24.052383 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-f2c4e823-9a21-4b7f-9860-a686e4b2c580 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61f61b293d3e43e2a2bf07a7c50ceb60'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:24.062367 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-f2c4e823-9a21-4b7f-9860-a686e4b2c580 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61f61b293d3e43e2a2bf07a7c50ceb60'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:24.066243 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16609d5-9d59-4892-b954-2365ebcc0651 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:24.067910 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-c16609d5-9d59-4892-b954-2365ebcc0651 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:24.068942 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1276/2562] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:21:23 2022] DELETE /identity/v3/users/61f61b293d3e43e2a2bf07a7c50ceb60 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:21:24.128378 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16609d5-9d59-4892-b954-2365ebcc0651 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:24.181242 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bb8f01-4f51-46a0-87f8-f41efc61ead2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:24.182145 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb8f01-4f51-46a0-87f8-f41efc61ead2 service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:24.182536 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb8f01-4f51-46a0-87f8-f41efc61ead2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:24.182901 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb8f01-4f51-46a0-87f8-f41efc61ead2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:24.184843 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-71bb8f01-4f51-46a0-87f8-f41efc61ead2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:24.257114 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bb8f01-4f51-46a0-87f8-f41efc61ead2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:24.259461 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bb8f01-4f51-46a0-87f8-f41efc61ead2 service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:24.269797 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1287/2563] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:21:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 01:21:24.351748 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c4e823-9a21-4b7f-9860-a686e4b2c580 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:24.354038 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-f2c4e823-9a21-4b7f-9860-a686e4b2c580 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:24.368683 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c4e823-9a21-4b7f-9860-a686e4b2c580 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:24.374104 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1c5a20-52b4-4202-a943-50fc9834ba02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:24.375131 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c5a20-52b4-4202-a943-50fc9834ba02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:24.375282 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c5a20-52b4-4202-a943-50fc9834ba02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:24.375639 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c5a20-52b4-4202-a943-50fc9834ba02 admin admin] PATH_INFO: `/v3/projects/24802648cdcf47f3918bd26d314755de` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:24.376984 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1c5a20-52b4-4202-a943-50fc9834ba02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24802648cdcf47f3918bd26d314755de)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:24.379043 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1c5a20-52b4-4202-a943-50fc9834ba02 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:24.438980 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-ab1c5a20-52b4-4202-a943-50fc9834ba02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24802648cdcf47f3918bd26d314755de', 'request_id': 'req-ab1c5a20-52b4-4202-a943-50fc9834ba02'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:24.449783 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-ab1c5a20-52b4-4202-a943-50fc9834ba02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24802648cdcf47f3918bd26d314755de', 'request_id': 'req-ab1c5a20-52b4-4202-a943-50fc9834ba02'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:24.457580 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-ab1c5a20-52b4-4202-a943-50fc9834ba02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24802648cdcf47f3918bd26d314755de', 'request_id': 'req-ab1c5a20-52b4-4202-a943-50fc9834ba02'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:24.472234 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-ab1c5a20-52b4-4202-a943-50fc9834ba02 admin admin] The token cache is being invalidate because project 24802648cdcf47f3918bd26d314755de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81661) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:21:24.472431 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-ab1c5a20-52b4-4202-a943-50fc9834ba02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:24.473539 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-ab1c5a20-52b4-4202-a943-50fc9834ba02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:24.475036 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1277/2564] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:21:24 2022] DELETE /identity/v3/projects/24802648cdcf47f3918bd26d314755de => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:21:24.480047 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a82753-eeb0-4e27-9ae9-103377b08b9a None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:24.480571 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a82753-eeb0-4e27-9ae9-103377b08b9a None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:24.480682 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a82753-eeb0-4e27-9ae9-103377b08b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:24.485839 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:21:24.541921 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-00a82753-eeb0-4e27-9ae9-103377b08b9a None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:21:24.543137 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-00a82753-eeb0-4e27-9ae9-103377b08b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:24.607490 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1288/2565] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:21:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:21:24.615362 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b3fef-2166-4150-b3b7-987da72cf7d7 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:24.615643 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b3fef-2166-4150-b3b7-987da72cf7d7 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:24.615918 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b3fef-2166-4150-b3b7-987da72cf7d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:24.639485 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-f93b3fef-2166-4150-b3b7-987da72cf7d7 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:21:24.640868 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-f93b3fef-2166-4150-b3b7-987da72cf7d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:24.648943 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1278/2566] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:21:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:21:24.653251 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a82753-eeb0-4e27-9ae9-103377b08b9a None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:24.663451 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a82753-eeb0-4e27-9ae9-103377b08b9a None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:24.668765 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfa9a32-6759-411a-88a7-8956a136959a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:24.669365 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa9a32-6759-411a-88a7-8956a136959a admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:24.669670 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa9a32-6759-411a-88a7-8956a136959a admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:24.669942 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa9a32-6759-411a-88a7-8956a136959a admin admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:24.670592 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfa9a32-6759-411a-88a7-8956a136959a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:21:24.670919 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfa9a32-6759-411a-88a7-8956a136959a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:24.672529 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfa9a32-6759-411a-88a7-8956a136959a admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:24.685201 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1289/2567] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:21:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:21:24.689685 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b3fef-2166-4150-b3b7-987da72cf7d7 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:24.699265 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b3fef-2166-4150-b3b7-987da72cf7d7 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:24.704011 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bbc969-0ab6-479c-b1f5-a7395a1c38fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:24.704635 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bbc969-0ab6-479c-b1f5-a7395a1c38fd admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:24.704913 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bbc969-0ab6-479c-b1f5-a7395a1c38fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:24.705188 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bbc969-0ab6-479c-b1f5-a7395a1c38fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:24.705813 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bbc969-0ab6-479c-b1f5-a7395a1c38fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:24.707772 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bbc969-0ab6-479c-b1f5-a7395a1c38fd admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:24.740126 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1279/2568] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:21:24 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:21:24.741155 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfa9a32-6759-411a-88a7-8956a136959a admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:24.753077 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfa9a32-6759-411a-88a7-8956a136959a admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:24.814844 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f65dac5-e3ef-4e8c-923f-c76067cafe6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:24.815457 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f65dac5-e3ef-4e8c-923f-c76067cafe6a admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:24.815735 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f65dac5-e3ef-4e8c-923f-c76067cafe6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:24.816006 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f65dac5-e3ef-4e8c-923f-c76067cafe6a admin admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:24.816696 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f65dac5-e3ef-4e8c-923f-c76067cafe6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:24.818623 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f65dac5-e3ef-4e8c-923f-c76067cafe6a admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:24.844168 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1290/2569] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:21:24 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 01:21:24.845908 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bbc969-0ab6-479c-b1f5-a7395a1c38fd admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:24.853185 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:21:24.857278 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bbc969-0ab6-479c-b1f5-a7395a1c38fd admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:24.863717 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ef5c3-c815-40e4-aa4c-a1f51be86aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:24.864561 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ef5c3-c815-40e4-aa4c-a1f51be86aed admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:24.865026 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ef5c3-c815-40e4-aa4c-a1f51be86aed admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:24.865423 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ef5c3-c815-40e4-aa4c-a1f51be86aed admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:24.866280 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9ef5c3-c815-40e4-aa4c-a1f51be86aed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:24.868176 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9ef5c3-c815-40e4-aa4c-a1f51be86aed admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:24.878630 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1280/2570] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:21:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 01:21:24.882435 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f65dac5-e3ef-4e8c-923f-c76067cafe6a admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:24.893464 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f65dac5-e3ef-4e8c-923f-c76067cafe6a admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:24.898134 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ecf820-1900-477f-a5ec-fe0825e29401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:24.898729 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ecf820-1900-477f-a5ec-fe0825e29401 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:24.899009 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ecf820-1900-477f-a5ec-fe0825e29401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:24.899341 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ecf820-1900-477f-a5ec-fe0825e29401 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:24.900019 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ecf820-1900-477f-a5ec-fe0825e29401 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:24.901649 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ecf820-1900-477f-a5ec-fe0825e29401 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:24.912655 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1291/2571] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:21:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:21:24.916873 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ef5c3-c815-40e4-aa4c-a1f51be86aed admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:24.927382 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ef5c3-c815-40e4-aa4c-a1f51be86aed admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:24.933324 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d78724-0d66-4a68-8a8f-c8e5e546cbef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:24.934211 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d78724-0d66-4a68-8a8f-c8e5e546cbef admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:24.934711 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d78724-0d66-4a68-8a8f-c8e5e546cbef admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:24.935183 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d78724-0d66-4a68-8a8f-c8e5e546cbef admin admin] PATH_INFO: `/v3/projects/ba22309ab8c1439d9c14ac3ee0a3aa3b/users/b3df60f66586419e97c490447193ea8a/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:24.950142 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d78724-0d66-4a68-8a8f-c8e5e546cbef admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba22309ab8c1439d9c14ac3ee0a3aa3b, user_id=b3df60f66586419e97c490447193ea8a, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:24.952869 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d78724-0d66-4a68-8a8f-c8e5e546cbef admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:24.968307 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1281/2572] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:21:24 2022] PUT /identity/v3/projects/ba22309ab8c1439d9c14ac3ee0a3aa3b/users/b3df60f66586419e97c490447193ea8a/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:21:24.979120 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8de6e-daa1-48d6-b0b4-309ff028e507 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:24.979410 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8de6e-daa1-48d6-b0b4-309ff028e507 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:24.979715 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8de6e-daa1-48d6-b0b4-309ff028e507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:25.007841 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-2ce8de6e-daa1-48d6-b0b4-309ff028e507 None None] MFA Rules not processed for user `b3df60f66586419e97c490447193ea8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:21:25.009408 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-2ce8de6e-daa1-48d6-b0b4-309ff028e507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:25.045299 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:21:25.079029 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1292/2573] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:21:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:21:25.083632 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d78724-0d66-4a68-8a8f-c8e5e546cbef admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:25.148199 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d78724-0d66-4a68-8a8f-c8e5e546cbef admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:25.155703 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5905b814-c7a4-45e6-bb41-e15f2b0af751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:25.156431 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5905b814-c7a4-45e6-bb41-e15f2b0af751 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:25.156698 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5905b814-c7a4-45e6-bb41-e15f2b0af751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:25.156972 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5905b814-c7a4-45e6-bb41-e15f2b0af751 admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:25.157627 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5905b814-c7a4-45e6-bb41-e15f2b0af751 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:25.159740 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5905b814-c7a4-45e6-bb41-e15f2b0af751 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:25.192594 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1282/2574] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:21:25 2022] POST /identity/v3/projects => generated 387 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:21:25.196576 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce8de6e-daa1-48d6-b0b4-309ff028e507 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:25.207744 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce8de6e-daa1-48d6-b0b4-309ff028e507 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:25.279253 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e0fcd5-99a1-4fb9-9b3d-fbcb18bb8f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:25.280174 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0fcd5-99a1-4fb9-9b3d-fbcb18bb8f4d admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:25.280631 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0fcd5-99a1-4fb9-9b3d-fbcb18bb8f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:25.281078 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0fcd5-99a1-4fb9-9b3d-fbcb18bb8f4d admin admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:25.281896 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e0fcd5-99a1-4fb9-9b3d-fbcb18bb8f4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:25.283863 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e0fcd5-99a1-4fb9-9b3d-fbcb18bb8f4d admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:25.311863 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1293/2575] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:21:25 2022] POST /identity/v3/users => generated 300 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 01:21:25.313113 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5905b814-c7a4-45e6-bb41-e15f2b0af751 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:25.330652 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5905b814-c7a4-45e6-bb41-e15f2b0af751 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:25.338374 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4656af69-390a-43e0-bd3b-c5e9da03247f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:25.339031 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4656af69-390a-43e0-bd3b-c5e9da03247f admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:25.339312 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4656af69-390a-43e0-bd3b-c5e9da03247f admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:25.339584 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4656af69-390a-43e0-bd3b-c5e9da03247f admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:25.340329 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4656af69-390a-43e0-bd3b-c5e9da03247f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:25.343021 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4656af69-390a-43e0-bd3b-c5e9da03247f admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:25.355589 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1283/2576] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:21:25 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:21:25.359663 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e0fcd5-99a1-4fb9-9b3d-fbcb18bb8f4d admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:25.370920 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e0fcd5-99a1-4fb9-9b3d-fbcb18bb8f4d admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:25.375936 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f687e-480a-4e46-8a88-a7b483cf2b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:25.376706 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f687e-480a-4e46-8a88-a7b483cf2b0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:25.377053 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f687e-480a-4e46-8a88-a7b483cf2b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:25.377393 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f687e-480a-4e46-8a88-a7b483cf2b0e admin admin] PATH_INFO: `/v3/projects/5635246347294d04ab92a2aefd3d9c61/users/d65bd11d10dc4131b478da1b42231f73/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:25.393742 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213f687e-480a-4e46-8a88-a7b483cf2b0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5635246347294d04ab92a2aefd3d9c61, user_id=d65bd11d10dc4131b478da1b42231f73, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:25.396568 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213f687e-480a-4e46-8a88-a7b483cf2b0e admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:25.411045 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1294/2577] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:21:25 2022] PUT /identity/v3/projects/5635246347294d04ab92a2aefd3d9c61/users/d65bd11d10dc4131b478da1b42231f73/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:21:25.415190 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4656af69-390a-43e0-bd3b-c5e9da03247f admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:25.425526 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4656af69-390a-43e0-bd3b-c5e9da03247f admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:25.487052 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f9af4f-4c25-4fce-8970-39d86b35064e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:25.487559 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9af4f-4c25-4fce-8970-39d86b35064e admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:25.487838 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9af4f-4c25-4fce-8970-39d86b35064e admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:25.488110 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9af4f-4c25-4fce-8970-39d86b35064e admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:25.488780 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f9af4f-4c25-4fce-8970-39d86b35064e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:25.490388 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f9af4f-4c25-4fce-8970-39d86b35064e admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:25.501384 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1284/2578] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:21:25 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:21:25.506519 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f687e-480a-4e46-8a88-a7b483cf2b0e admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:25.521620 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f687e-480a-4e46-8a88-a7b483cf2b0e admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:25.529635 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9e6ec6-bdcf-4aef-a2d2-091fa5838572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:25.530265 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e6ec6-bdcf-4aef-a2d2-091fa5838572 admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:25.530545 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e6ec6-bdcf-4aef-a2d2-091fa5838572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:25.530814 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e6ec6-bdcf-4aef-a2d2-091fa5838572 admin admin] PATH_INFO: `/v3/projects/5635246347294d04ab92a2aefd3d9c61/users/d65bd11d10dc4131b478da1b42231f73/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:25.532927 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9e6ec6-bdcf-4aef-a2d2-091fa5838572 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5635246347294d04ab92a2aefd3d9c61, user_id=d65bd11d10dc4131b478da1b42231f73, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:25.535910 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9e6ec6-bdcf-4aef-a2d2-091fa5838572 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:25.546898 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1295/2579] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:21:25 2022] PUT /identity/v3/projects/5635246347294d04ab92a2aefd3d9c61/users/d65bd11d10dc4131b478da1b42231f73/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:21:25.558360 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2ceae-4a56-4aa3-94f8-6890a8b4cdd6 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:25.558653 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2ceae-4a56-4aa3-94f8-6890a8b4cdd6 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:25.558908 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2ceae-4a56-4aa3-94f8-6890a8b4cdd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:25.592577 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-c4e2ceae-4a56-4aa3-94f8-6890a8b4cdd6 None None] MFA Rules not processed for user `d65bd11d10dc4131b478da1b42231f73`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:21:25.593809 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-c4e2ceae-4a56-4aa3-94f8-6890a8b4cdd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:25.629834 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:21:25.678170 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1285/2580] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:21:25 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:21:25.689893 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9e6ec6-bdcf-4aef-a2d2-091fa5838572 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:25.691811 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-9e9e6ec6-bdcf-4aef-a2d2-091fa5838572 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:25.778801 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9e6ec6-bdcf-4aef-a2d2-091fa5838572 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:25.783117 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af839130-100f-449b-87bf-ee980d8638a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:25.784274 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af839130-100f-449b-87bf-ee980d8638a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:25.784631 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af839130-100f-449b-87bf-ee980d8638a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:25.784910 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af839130-100f-449b-87bf-ee980d8638a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:25.795685 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af839130-100f-449b-87bf-ee980d8638a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:25.797268 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af839130-100f-449b-87bf-ee980d8638a6 service cinder] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:25.808915 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1296/2581] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:21:25 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:21:26.558038 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e2ceae-4a56-4aa3-94f8-6890a8b4cdd6 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:26.626711 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e2ceae-4a56-4aa3-94f8-6890a8b4cdd6 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:26.631398 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bf97e0-ca59-45e9-9089-2ec4c833c2e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:26.632033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf97e0-ca59-45e9-9089-2ec4c833c2e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:26.632340 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf97e0-ca59-45e9-9089-2ec4c833c2e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:26.632612 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf97e0-ca59-45e9-9089-2ec4c833c2e1 admin admin] PATH_INFO: `/v3/users/b3df60f66586419e97c490447193ea8a` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:26.634716 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bf97e0-ca59-45e9-9089-2ec4c833c2e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3df60f66586419e97c490447193ea8a)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:26.637003 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bf97e0-ca59-45e9-9089-2ec4c833c2e1 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:26.686507 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-54bf97e0-ca59-45e9-9089-2ec4c833c2e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3df60f66586419e97c490447193ea8a'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:26.700693 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-54bf97e0-ca59-45e9-9089-2ec4c833c2e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3df60f66586419e97c490447193ea8a'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:26.708085 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-54bf97e0-ca59-45e9-9089-2ec4c833c2e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3df60f66586419e97c490447193ea8a'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:26.708751 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-54bf97e0-ca59-45e9-9089-2ec4c833c2e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3df60f66586419e97c490447193ea8a'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:26.717012 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-54bf97e0-ca59-45e9-9089-2ec4c833c2e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3df60f66586419e97c490447193ea8a'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:26.723041 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1286/2582] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:21:26 2022] DELETE /identity/v3/users/b3df60f66586419e97c490447193ea8a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:21:26.727922 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85bb9a-e6cd-48ac-bc23-2ed7f279875b None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:26.728448 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85bb9a-e6cd-48ac-bc23-2ed7f279875b None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:26.728656 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85bb9a-e6cd-48ac-bc23-2ed7f279875b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:26.748746 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-cb85bb9a-e6cd-48ac-bc23-2ed7f279875b None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:21:26.749954 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-cb85bb9a-e6cd-48ac-bc23-2ed7f279875b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:26.778903 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:21:26.822471 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1297/2583] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:21:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:21:26.833839 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bf97e0-ca59-45e9-9089-2ec4c833c2e1 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:26.836811 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-54bf97e0-ca59-45e9-9089-2ec4c833c2e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:26.898832 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:21:26.914638 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bf97e0-ca59-45e9-9089-2ec4c833c2e1 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:26.921406 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7dae34-1270-4293-af08-455e08f46bf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:26.922113 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7dae34-1270-4293-af08-455e08f46bf4 service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:26.922465 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7dae34-1270-4293-af08-455e08f46bf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:26.923834 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7dae34-1270-4293-af08-455e08f46bf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:26.935989 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7dae34-1270-4293-af08-455e08f46bf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:26.937573 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7dae34-1270-4293-af08-455e08f46bf4 service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:26.949902 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1287/2584] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:21:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:21:27.429235 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb85bb9a-e6cd-48ac-bc23-2ed7f279875b None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:27.432064 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-cb85bb9a-e6cd-48ac-bc23-2ed7f279875b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:27.452397 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb85bb9a-e6cd-48ac-bc23-2ed7f279875b None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:27.471874 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d33e73-db34-485e-876b-86f32084346b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:27.472502 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d33e73-db34-485e-876b-86f32084346b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:27.472798 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d33e73-db34-485e-876b-86f32084346b admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:27.474785 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d33e73-db34-485e-876b-86f32084346b admin admin] PATH_INFO: `/v3/projects/ba22309ab8c1439d9c14ac3ee0a3aa3b` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:27.480413 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d33e73-db34-485e-876b-86f32084346b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba22309ab8c1439d9c14ac3ee0a3aa3b)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:27.483697 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d33e73-db34-485e-876b-86f32084346b admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:27.553921 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-22d33e73-db34-485e-876b-86f32084346b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba22309ab8c1439d9c14ac3ee0a3aa3b', 'request_id': 'req-22d33e73-db34-485e-876b-86f32084346b'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:27.564985 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-22d33e73-db34-485e-876b-86f32084346b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba22309ab8c1439d9c14ac3ee0a3aa3b', 'request_id': 'req-22d33e73-db34-485e-876b-86f32084346b'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:27.571543 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-22d33e73-db34-485e-876b-86f32084346b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba22309ab8c1439d9c14ac3ee0a3aa3b', 'request_id': 'req-22d33e73-db34-485e-876b-86f32084346b'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:27.587155 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-22d33e73-db34-485e-876b-86f32084346b admin admin] The token cache is being invalidate because project ba22309ab8c1439d9c14ac3ee0a3aa3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:21:27.587838 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-22d33e73-db34-485e-876b-86f32084346b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:27.588742 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-22d33e73-db34-485e-876b-86f32084346b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:27.593251 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1298/2585] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:21:27 2022] DELETE /identity/v3/projects/ba22309ab8c1439d9c14ac3ee0a3aa3b => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:21:27.594662 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7dae34-1270-4293-af08-455e08f46bf4 service neutron] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:27.623960 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7dae34-1270-4293-af08-455e08f46bf4 service neutron] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:27.663287 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d33e73-db34-485e-876b-86f32084346b admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:27.665340 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-22d33e73-db34-485e-876b-86f32084346b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:27.702404 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0bcf65-6c35-48c9-ae06-60e1f1110bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:27.703052 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0bcf65-6c35-48c9-ae06-60e1f1110bde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:27.703336 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0bcf65-6c35-48c9-ae06-60e1f1110bde admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:27.703609 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0bcf65-6c35-48c9-ae06-60e1f1110bde admin admin] PATH_INFO: `/v3/users/d65bd11d10dc4131b478da1b42231f73` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:27.704752 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0bcf65-6c35-48c9-ae06-60e1f1110bde admin admin] RBAC: Authorizing `identity:delete_user(user_id=d65bd11d10dc4131b478da1b42231f73)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:27.706672 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0bcf65-6c35-48c9-ae06-60e1f1110bde admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:27.735243 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d33e73-db34-485e-876b-86f32084346b admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:27.740681 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5825596a-96e6-4f33-a34a-36f47030e742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:27.741412 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825596a-96e6-4f33-a34a-36f47030e742 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:27.741688 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825596a-96e6-4f33-a34a-36f47030e742 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:27.741964 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825596a-96e6-4f33-a34a-36f47030e742 admin admin] PATH_INFO: `/v3/users/9f2e208a67d4443eb00c75c60c54173f` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:27.744232 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5825596a-96e6-4f33-a34a-36f47030e742 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f2e208a67d4443eb00c75c60c54173f)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:27.746164 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5825596a-96e6-4f33-a34a-36f47030e742 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:27.757966 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-be0bcf65-6c35-48c9-ae06-60e1f1110bde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd65bd11d10dc4131b478da1b42231f73'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:27.771634 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-be0bcf65-6c35-48c9-ae06-60e1f1110bde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd65bd11d10dc4131b478da1b42231f73'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:27.779845 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-be0bcf65-6c35-48c9-ae06-60e1f1110bde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd65bd11d10dc4131b478da1b42231f73'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:27.780770 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-be0bcf65-6c35-48c9-ae06-60e1f1110bde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd65bd11d10dc4131b478da1b42231f73'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:27.790586 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-be0bcf65-6c35-48c9-ae06-60e1f1110bde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd65bd11d10dc4131b478da1b42231f73'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:27.798858 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-5825596a-96e6-4f33-a34a-36f47030e742 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f2e208a67d4443eb00c75c60c54173f'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:27.799283 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1288/2586] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:21:27 2022] DELETE /identity/v3/users/d65bd11d10dc4131b478da1b42231f73 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:21:27.805311 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:21:27.814179 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-5825596a-96e6-4f33-a34a-36f47030e742 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f2e208a67d4443eb00c75c60c54173f'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:27.822205 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-5825596a-96e6-4f33-a34a-36f47030e742 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f2e208a67d4443eb00c75c60c54173f'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:27.829751 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-5825596a-96e6-4f33-a34a-36f47030e742 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f2e208a67d4443eb00c75c60c54173f'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:27.830405 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-5825596a-96e6-4f33-a34a-36f47030e742 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f2e208a67d4443eb00c75c60c54173f'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:27.836784 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1299/2587] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:21:27 2022] DELETE /identity/v3/users/9f2e208a67d4443eb00c75c60c54173f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:21:28.027628 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0bcf65-6c35-48c9-ae06-60e1f1110bde admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:28.029826 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-be0bcf65-6c35-48c9-ae06-60e1f1110bde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:28.102216 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5825596a-96e6-4f33-a34a-36f47030e742 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:28.104605 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-5825596a-96e6-4f33-a34a-36f47030e742 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:28.112520 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0bcf65-6c35-48c9-ae06-60e1f1110bde admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:28.125930 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:21:28.185839 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5825596a-96e6-4f33-a34a-36f47030e742 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:28.187105 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4659a396-d965-4286-92a6-9c53652991c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:28.187802 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4659a396-d965-4286-92a6-9c53652991c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:28.188171 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4659a396-d965-4286-92a6-9c53652991c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:28.188660 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4659a396-d965-4286-92a6-9c53652991c6 admin admin] PATH_INFO: `/v3/projects/34623908736a4e3d9f015fc9db90bd0d` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:28.189778 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4659a396-d965-4286-92a6-9c53652991c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34623908736a4e3d9f015fc9db90bd0d)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:28.190547 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba579801-7913-4f37-826c-24480c3589a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:28.191191 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba579801-7913-4f37-826c-24480c3589a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:28.191464 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba579801-7913-4f37-826c-24480c3589a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:28.191686 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4659a396-d965-4286-92a6-9c53652991c6 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:28.191797 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba579801-7913-4f37-826c-24480c3589a6 admin admin] PATH_INFO: `/v3/projects/5635246347294d04ab92a2aefd3d9c61` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:28.192884 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba579801-7913-4f37-826c-24480c3589a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5635246347294d04ab92a2aefd3d9c61)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:28.194784 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba579801-7913-4f37-826c-24480c3589a6 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:28.241232 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-4659a396-d965-4286-92a6-9c53652991c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34623908736a4e3d9f015fc9db90bd0d', 'request_id': 'req-4659a396-d965-4286-92a6-9c53652991c6'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:28.244118 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-ba579801-7913-4f37-826c-24480c3589a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5635246347294d04ab92a2aefd3d9c61', 'request_id': 'req-ba579801-7913-4f37-826c-24480c3589a6'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:28.249224 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-4659a396-d965-4286-92a6-9c53652991c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34623908736a4e3d9f015fc9db90bd0d', 'request_id': 'req-4659a396-d965-4286-92a6-9c53652991c6'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:28.250678 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-ba579801-7913-4f37-826c-24480c3589a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5635246347294d04ab92a2aefd3d9c61', 'request_id': 'req-ba579801-7913-4f37-826c-24480c3589a6'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:28.256129 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-4659a396-d965-4286-92a6-9c53652991c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34623908736a4e3d9f015fc9db90bd0d', 'request_id': 'req-4659a396-d965-4286-92a6-9c53652991c6'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:28.256465 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-ba579801-7913-4f37-826c-24480c3589a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5635246347294d04ab92a2aefd3d9c61', 'request_id': 'req-ba579801-7913-4f37-826c-24480c3589a6'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:28.267988 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-4659a396-d965-4286-92a6-9c53652991c6 admin admin] The token cache is being invalidate because project 34623908736a4e3d9f015fc9db90bd0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81661) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:21:28.268338 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-4659a396-d965-4286-92a6-9c53652991c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:28.268977 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-4659a396-d965-4286-92a6-9c53652991c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:28.269339 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-ba579801-7913-4f37-826c-24480c3589a6 admin admin] The token cache is being invalidate because project 5635246347294d04ab92a2aefd3d9c61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:21:28.269638 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-ba579801-7913-4f37-826c-24480c3589a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:28.270259 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-ba579801-7913-4f37-826c-24480c3589a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:28.274087 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1289/2588] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:21:28 2022] DELETE /identity/v3/projects/34623908736a4e3d9f015fc9db90bd0d => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:21:28.274292 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1300/2589] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:21:28 2022] DELETE /identity/v3/projects/5635246347294d04ab92a2aefd3d9c61 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:21:28.275637 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac5fd2-f34b-4614-b4e3-ffc4962ee66e None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:28.276559 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac5fd2-f34b-4614-b4e3-ffc4962ee66e None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:28.276922 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b4b38-1eb8-4725-9dfd-287c984f1d35 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:28.277033 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac5fd2-f34b-4614-b4e3-ffc4962ee66e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:28.277212 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b4b38-1eb8-4725-9dfd-287c984f1d35 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:28.277555 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b4b38-1eb8-4725-9dfd-287c984f1d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:28.298135 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-882b4b38-1eb8-4725-9dfd-287c984f1d35 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:21:28.299355 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-882b4b38-1eb8-4725-9dfd-287c984f1d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:28.299821 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-adac5fd2-f34b-4614-b4e3-ffc4962ee66e None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:21:28.301110 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-adac5fd2-f34b-4614-b4e3-ffc4962ee66e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:28.359261 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1301/2590] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:21:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:21:28.364525 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1290/2591] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:21:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:21:28.367165 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3392a-6718-476a-9016-252e5687979b None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:28.367458 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3392a-6718-476a-9016-252e5687979b None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:28.367736 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3392a-6718-476a-9016-252e5687979b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:28.372533 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7472ea-5f8e-4525-b67a-7693acdf56ce None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:28.372818 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7472ea-5f8e-4525-b67a-7693acdf56ce None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:28.373095 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7472ea-5f8e-4525-b67a-7693acdf56ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:28.375794 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:21:28.390982 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-73b3392a-6718-476a-9016-252e5687979b None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:21:28.392259 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-73b3392a-6718-476a-9016-252e5687979b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:28.394315 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-9a7472ea-5f8e-4525-b67a-7693acdf56ce None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:21:28.395484 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-9a7472ea-5f8e-4525-b67a-7693acdf56ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:28.401309 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1302/2592] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:21:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:21:28.404502 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1291/2593] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:21:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:21:28.405914 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b3392a-6718-476a-9016-252e5687979b None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:28.408665 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7472ea-5f8e-4525-b67a-7693acdf56ce None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:28.416753 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b3392a-6718-476a-9016-252e5687979b None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:28.418789 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7472ea-5f8e-4525-b67a-7693acdf56ce None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:28.421522 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa9ea7-5c7e-4236-918b-bbfb128df308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:28.422252 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa9ea7-5c7e-4236-918b-bbfb128df308 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:28.422558 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa9ea7-5c7e-4236-918b-bbfb128df308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:28.422796 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa9ea7-5c7e-4236-918b-bbfb128df308 admin admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:28.423492 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fa9ea7-5c7e-4236-918b-bbfb128df308 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:21:28.423753 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7acac7e-8f03-465e-8595-28909fad07e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:28.423884 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fa9ea7-5c7e-4236-918b-bbfb128df308 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:28.424370 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7acac7e-8f03-465e-8595-28909fad07e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:28.424648 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7acac7e-8f03-465e-8595-28909fad07e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:28.424916 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7acac7e-8f03-465e-8595-28909fad07e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:28.425462 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fa9ea7-5c7e-4236-918b-bbfb128df308 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:28.425576 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7acac7e-8f03-465e-8595-28909fad07e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:21:28.425889 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7acac7e-8f03-465e-8595-28909fad07e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:28.427482 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7acac7e-8f03-465e-8595-28909fad07e3 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:28.437767 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1303/2594] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:21:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:21:28.443055 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1292/2595] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:21:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:21:28.443193 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa9ea7-5c7e-4236-918b-bbfb128df308 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:28.443930 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7acac7e-8f03-465e-8595-28909fad07e3 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:28.452995 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa9ea7-5c7e-4236-918b-bbfb128df308 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:28.455412 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7acac7e-8f03-465e-8595-28909fad07e3 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:28.457682 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e168755-1f58-4749-9ee6-9b06d97d23fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:28.458272 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e168755-1f58-4749-9ee6-9b06d97d23fd admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:28.458553 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e168755-1f58-4749-9ee6-9b06d97d23fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:28.458828 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e168755-1f58-4749-9ee6-9b06d97d23fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:28.459471 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e168755-1f58-4749-9ee6-9b06d97d23fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:28.460339 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b968e180-81ab-49d5-b13b-578308870e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:28.460914 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b968e180-81ab-49d5-b13b-578308870e09 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:28.461200 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b968e180-81ab-49d5-b13b-578308870e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:28.461473 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b968e180-81ab-49d5-b13b-578308870e09 admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:28.461662 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e168755-1f58-4749-9ee6-9b06d97d23fd admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:28.462712 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b968e180-81ab-49d5-b13b-578308870e09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:28.464585 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b968e180-81ab-49d5-b13b-578308870e09 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:28.490276 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1304/2596] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:21:28 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:21:28.494458 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e168755-1f58-4749-9ee6-9b06d97d23fd admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:28.500419 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1293/2597] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:21:28 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 01:21:28.502420 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b968e180-81ab-49d5-b13b-578308870e09 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:28.507057 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:21:28.563084 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e168755-1f58-4749-9ee6-9b06d97d23fd admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:28.567880 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5291052-2e39-4a4d-9044-c35188fc0cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:28.568521 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5291052-2e39-4a4d-9044-c35188fc0cc3 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:28.568804 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5291052-2e39-4a4d-9044-c35188fc0cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:28.569079 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5291052-2e39-4a4d-9044-c35188fc0cc3 admin admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:28.569716 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5291052-2e39-4a4d-9044-c35188fc0cc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:28.572399 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5291052-2e39-4a4d-9044-c35188fc0cc3 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:28.572546 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b968e180-81ab-49d5-b13b-578308870e09 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:28.577036 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca39cc6-d2a5-47e9-ac03-ef8755b71b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:28.577657 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca39cc6-d2a5-47e9-ac03-ef8755b71b8c admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:28.577974 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca39cc6-d2a5-47e9-ac03-ef8755b71b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:28.578304 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca39cc6-d2a5-47e9-ac03-ef8755b71b8c admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:28.579182 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca39cc6-d2a5-47e9-ac03-ef8755b71b8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:28.581524 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca39cc6-d2a5-47e9-ac03-ef8755b71b8c admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:28.599648 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1305/2598] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:21:28 2022] POST /identity/v3/users => generated 288 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 01:21:28.604065 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5291052-2e39-4a4d-9044-c35188fc0cc3 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:28.610225 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1294/2599] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:21:28 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:21:28.611278 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca39cc6-d2a5-47e9-ac03-ef8755b71b8c admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:28.616020 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5291052-2e39-4a4d-9044-c35188fc0cc3 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:28.621090 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca39cc6-d2a5-47e9-ac03-ef8755b71b8c admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:28.625214 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a66fec-8e2d-46a2-9a19-2b201729b105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:28.625826 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a66fec-8e2d-46a2-9a19-2b201729b105 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:28.625958 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f91cc54-fda4-481f-86f3-f7876c876dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:28.626170 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a66fec-8e2d-46a2-9a19-2b201729b105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:28.626449 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a66fec-8e2d-46a2-9a19-2b201729b105 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:28.626553 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f91cc54-fda4-481f-86f3-f7876c876dd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:28.626829 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f91cc54-fda4-481f-86f3-f7876c876dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:28.627099 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f91cc54-fda4-481f-86f3-f7876c876dd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:28.627197 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a66fec-8e2d-46a2-9a19-2b201729b105 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:28.627764 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f91cc54-fda4-481f-86f3-f7876c876dd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:28.628993 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a66fec-8e2d-46a2-9a19-2b201729b105 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:28.629391 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f91cc54-fda4-481f-86f3-f7876c876dd4 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:28.640676 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1306/2600] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:21:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 01:21:28.643699 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1295/2601] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:21:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 01:21:28.647350 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a66fec-8e2d-46a2-9a19-2b201729b105 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:28.647508 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f91cc54-fda4-481f-86f3-f7876c876dd4 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:28.658257 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a66fec-8e2d-46a2-9a19-2b201729b105 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:28.658446 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f91cc54-fda4-481f-86f3-f7876c876dd4 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:28.662989 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c591f-0bfd-4b4a-9741-69d2e3b3cd0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:28.663140 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d45386-3efc-4bb8-baf8-ba8d40c77b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:28.663635 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c591f-0bfd-4b4a-9741-69d2e3b3cd0c admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:28.663759 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d45386-3efc-4bb8-baf8-ba8d40c77b05 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:28.663915 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d45386-3efc-4bb8-baf8-ba8d40c77b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:28.664022 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c591f-0bfd-4b4a-9741-69d2e3b3cd0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:28.664282 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c591f-0bfd-4b4a-9741-69d2e3b3cd0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:28.664457 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d45386-3efc-4bb8-baf8-ba8d40c77b05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:28.664953 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c591f-0bfd-4b4a-9741-69d2e3b3cd0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:28.665317 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d45386-3efc-4bb8-baf8-ba8d40c77b05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:28.666660 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c591f-0bfd-4b4a-9741-69d2e3b3cd0c admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:28.666945 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d45386-3efc-4bb8-baf8-ba8d40c77b05 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:28.677447 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1307/2602] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:21:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 01:21:28.678151 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1296/2603] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:21:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 01:21:28.683091 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c591f-0bfd-4b4a-9741-69d2e3b3cd0c admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:28.683257 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d45386-3efc-4bb8-baf8-ba8d40c77b05 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:28.693046 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d45386-3efc-4bb8-baf8-ba8d40c77b05 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:28.693240 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c591f-0bfd-4b4a-9741-69d2e3b3cd0c admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:28.698313 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea564152-b2b6-49e5-8e6b-e0a92ca31b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:28.698455 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a0bb0b-ec29-473f-a6ec-970d29dd36ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:28.698947 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea564152-b2b6-49e5-8e6b-e0a92ca31b62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:28.699072 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0bb0b-ec29-473f-a6ec-970d29dd36ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:28.699226 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea564152-b2b6-49e5-8e6b-e0a92ca31b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:28.699325 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0bb0b-ec29-473f-a6ec-970d29dd36ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:28.699496 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea564152-b2b6-49e5-8e6b-e0a92ca31b62 admin admin] PATH_INFO: `/v3/projects/89fc5b93a67d4b97be3cbcb8cdd7e3c3/users/49a3d57540ad4814b196a655b306eb2d/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:28.699657 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0bb0b-ec29-473f-a6ec-970d29dd36ad admin admin] PATH_INFO: `/v3/projects/76e39b45fb864237a4bc22be6fc0aec8/users/a5ea37cfe6204b2dbd825e702d636b93/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:28.714901 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea564152-b2b6-49e5-8e6b-e0a92ca31b62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89fc5b93a67d4b97be3cbcb8cdd7e3c3, user_id=49a3d57540ad4814b196a655b306eb2d, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:28.715966 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a0bb0b-ec29-473f-a6ec-970d29dd36ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=76e39b45fb864237a4bc22be6fc0aec8, user_id=a5ea37cfe6204b2dbd825e702d636b93, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:28.717741 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea564152-b2b6-49e5-8e6b-e0a92ca31b62 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:28.718704 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a0bb0b-ec29-473f-a6ec-970d29dd36ad admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:28.732005 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1297/2604] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:21:28 2022] PUT /identity/v3/projects/89fc5b93a67d4b97be3cbcb8cdd7e3c3/users/49a3d57540ad4814b196a655b306eb2d/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:21:28.732315 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1308/2605] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:21:28 2022] PUT /identity/v3/projects/76e39b45fb864237a4bc22be6fc0aec8/users/a5ea37cfe6204b2dbd825e702d636b93/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:21:28.741812 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdc7889-c9e8-48d0-9d2a-aa019160902e None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:28.742097 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdc7889-c9e8-48d0-9d2a-aa019160902e None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:28.742599 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdc7889-c9e8-48d0-9d2a-aa019160902e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:28.744420 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f8eae-a156-42ac-a823-7843dcb52267 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:28.744945 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f8eae-a156-42ac-a823-7843dcb52267 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:28.745060 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f8eae-a156-42ac-a823-7843dcb52267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:28.772434 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-1fdc7889-c9e8-48d0-9d2a-aa019160902e None None] MFA Rules not processed for user `49a3d57540ad4814b196a655b306eb2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:21:28.773201 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-346f8eae-a156-42ac-a823-7843dcb52267 None None] MFA Rules not processed for user `a5ea37cfe6204b2dbd825e702d636b93`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:21:28.773691 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-1fdc7889-c9e8-48d0-9d2a-aa019160902e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:28.774594 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-346f8eae-a156-42ac-a823-7843dcb52267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:28.830722 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:21:28.840976 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1298/2606] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:21:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:21:28.845873 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1309/2607] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:21:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:21:28.852014 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdc7889-c9e8-48d0-9d2a-aa019160902e None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:28.853580 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-1fdc7889-c9e8-48d0-9d2a-aa019160902e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:28.856452 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346f8eae-a156-42ac-a823-7843dcb52267 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:28.857861 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-346f8eae-a156-42ac-a823-7843dcb52267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:28.896704 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:21:28.915840 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdc7889-c9e8-48d0-9d2a-aa019160902e None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:28.919613 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb1cc34-21a1-4244-ade8-b81a6969de79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:28.920255 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb1cc34-21a1-4244-ade8-b81a6969de79 service cinder] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:28.920541 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb1cc34-21a1-4244-ade8-b81a6969de79 service cinder] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:28.920816 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb1cc34-21a1-4244-ade8-b81a6969de79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:28.929486 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb1cc34-21a1-4244-ade8-b81a6969de79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:28.931469 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb1cc34-21a1-4244-ade8-b81a6969de79 service cinder] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:28.940596 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1299/2608] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:21:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:21:28.972936 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346f8eae-a156-42ac-a823-7843dcb52267 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:28.982569 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1683b10a-e411-4f01-be7f-d3a79f1c68bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae643cfdcb164aa9b474747cd21aa116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:28.983547 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1683b10a-e411-4f01-be7f-d3a79f1c68bd service nova] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:28.984019 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1683b10a-e411-4f01-be7f-d3a79f1c68bd service nova] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:28.984520 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1683b10a-e411-4f01-be7f-d3a79f1c68bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:28.997307 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1683b10a-e411-4f01-be7f-d3a79f1c68bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:28.999966 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1683b10a-e411-4f01-be7f-d3a79f1c68bd service nova] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:29.014032 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1310/2609] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:21:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:21:32.620758 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb1cc34-21a1-4244-ade8-b81a6969de79 service cinder] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:32.622174 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-4eb1cc34-21a1-4244-ade8-b81a6969de79 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:32.715482 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb1cc34-21a1-4244-ade8-b81a6969de79 service cinder] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:32.720595 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854f82d4-4bc8-4a31-bf1d-dee577462d4e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '301d30bbd3de4cc69e96bb12729442c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:32.721190 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854f82d4-4bc8-4a31-bf1d-dee577462d4e service placement] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:32.721469 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854f82d4-4bc8-4a31-bf1d-dee577462d4e service placement] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:32.721738 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854f82d4-4bc8-4a31-bf1d-dee577462d4e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:32.723457 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-854f82d4-4bc8-4a31-bf1d-dee577462d4e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:32.735291 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854f82d4-4bc8-4a31-bf1d-dee577462d4e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:32.736921 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854f82d4-4bc8-4a31-bf1d-dee577462d4e service placement] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:32.747681 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1300/2610] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:21:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:21:39.007038 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=49 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:21:41.756511 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1683b10a-e411-4f01-be7f-d3a79f1c68bd service nova] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:41.820454 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1683b10a-e411-4f01-be7f-d3a79f1c68bd service nova] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:41.825022 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd0c93f-8301-487c-9bd1-68b09ec55fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:41.825648 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd0c93f-8301-487c-9bd1-68b09ec55fab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:41.825932 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd0c93f-8301-487c-9bd1-68b09ec55fab admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:41.826208 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd0c93f-8301-487c-9bd1-68b09ec55fab admin admin] PATH_INFO: `/v3/users/a5ea37cfe6204b2dbd825e702d636b93` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:41.827335 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd0c93f-8301-487c-9bd1-68b09ec55fab admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5ea37cfe6204b2dbd825e702d636b93)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:41.829317 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd0c93f-8301-487c-9bd1-68b09ec55fab admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:41.876781 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-edd0c93f-8301-487c-9bd1-68b09ec55fab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5ea37cfe6204b2dbd825e702d636b93'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:41.889839 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-edd0c93f-8301-487c-9bd1-68b09ec55fab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5ea37cfe6204b2dbd825e702d636b93'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:41.896825 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-edd0c93f-8301-487c-9bd1-68b09ec55fab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5ea37cfe6204b2dbd825e702d636b93'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:41.903947 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-edd0c93f-8301-487c-9bd1-68b09ec55fab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5ea37cfe6204b2dbd825e702d636b93'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:41.904600 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-edd0c93f-8301-487c-9bd1-68b09ec55fab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5ea37cfe6204b2dbd825e702d636b93'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:41.910649 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1311/2611] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:21:41 2022] DELETE /identity/v3/users/a5ea37cfe6204b2dbd825e702d636b93 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:21:41.916023 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56340b1d-7d2b-4d80-be4c-cac6b4584475 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:41.916543 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56340b1d-7d2b-4d80-be4c-cac6b4584475 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:41.916656 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56340b1d-7d2b-4d80-be4c-cac6b4584475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:41.936818 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-56340b1d-7d2b-4d80-be4c-cac6b4584475 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:21:41.937963 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-56340b1d-7d2b-4d80-be4c-cac6b4584475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:41.996441 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1301/2612] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:21:42.006064 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd0c93f-8301-487c-9bd1-68b09ec55fab admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:42.007469 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-edd0c93f-8301-487c-9bd1-68b09ec55fab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:42.091769 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd0c93f-8301-487c-9bd1-68b09ec55fab admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:42.095454 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1943a0-4f8f-426d-936b-c755a4dfcd7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:42.096268 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1943a0-4f8f-426d-936b-c755a4dfcd7d service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:42.096523 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1943a0-4f8f-426d-936b-c755a4dfcd7d service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:42.096928 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1943a0-4f8f-426d-936b-c755a4dfcd7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:42.107471 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1943a0-4f8f-426d-936b-c755a4dfcd7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:42.108962 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1943a0-4f8f-426d-936b-c755a4dfcd7d service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:42.119611 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1312/2613] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:21:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:21:42.393760 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56340b1d-7d2b-4d80-be4c-cac6b4584475 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:42.395583 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-56340b1d-7d2b-4d80-be4c-cac6b4584475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:42.418422 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56340b1d-7d2b-4d80-be4c-cac6b4584475 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:42.423064 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7951ed63-3689-4a6e-a15e-c4c8371fc511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:42.423693 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951ed63-3689-4a6e-a15e-c4c8371fc511 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:42.423971 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951ed63-3689-4a6e-a15e-c4c8371fc511 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:42.424274 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951ed63-3689-4a6e-a15e-c4c8371fc511 admin admin] PATH_INFO: `/v3/projects/76e39b45fb864237a4bc22be6fc0aec8` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:42.425551 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7951ed63-3689-4a6e-a15e-c4c8371fc511 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76e39b45fb864237a4bc22be6fc0aec8)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:42.427485 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7951ed63-3689-4a6e-a15e-c4c8371fc511 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:42.471945 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:21:42.477995 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-7951ed63-3689-4a6e-a15e-c4c8371fc511 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76e39b45fb864237a4bc22be6fc0aec8', 'request_id': 'req-7951ed63-3689-4a6e-a15e-c4c8371fc511'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:42.485800 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-7951ed63-3689-4a6e-a15e-c4c8371fc511 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76e39b45fb864237a4bc22be6fc0aec8', 'request_id': 'req-7951ed63-3689-4a6e-a15e-c4c8371fc511'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:42.492503 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-7951ed63-3689-4a6e-a15e-c4c8371fc511 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76e39b45fb864237a4bc22be6fc0aec8', 'request_id': 'req-7951ed63-3689-4a6e-a15e-c4c8371fc511'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:42.502544 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-7951ed63-3689-4a6e-a15e-c4c8371fc511 admin admin] The token cache is being invalidate because project 76e39b45fb864237a4bc22be6fc0aec8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81661) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:21:42.502782 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-7951ed63-3689-4a6e-a15e-c4c8371fc511 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:42.503402 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-7951ed63-3689-4a6e-a15e-c4c8371fc511 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:42.504905 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1302/2614] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:21:42 2022] DELETE /identity/v3/projects/76e39b45fb864237a4bc22be6fc0aec8 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:21:42.511506 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc48b13-b5ec-4efb-8e75-6b36cc10e61d None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:42.511790 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc48b13-b5ec-4efb-8e75-6b36cc10e61d None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:42.512067 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc48b13-b5ec-4efb-8e75-6b36cc10e61d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:42.519805 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:21:42.533827 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-abc48b13-b5ec-4efb-8e75-6b36cc10e61d None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:21:42.535409 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-abc48b13-b5ec-4efb-8e75-6b36cc10e61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:42.594748 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1313/2615] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:21:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:21:42.602415 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae975bf3-3e9f-47ca-b25f-c462ec7cb349 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:42.602700 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae975bf3-3e9f-47ca-b25f-c462ec7cb349 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:42.603002 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae975bf3-3e9f-47ca-b25f-c462ec7cb349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:42.622973 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-ae975bf3-3e9f-47ca-b25f-c462ec7cb349 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:21:42.624206 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-ae975bf3-3e9f-47ca-b25f-c462ec7cb349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:42.632379 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1303/2616] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:21:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:21:42.637492 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc48b13-b5ec-4efb-8e75-6b36cc10e61d None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:42.647321 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc48b13-b5ec-4efb-8e75-6b36cc10e61d None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:42.652854 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d65cfbe-3e52-485f-8e58-bb5d725bda09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:42.653451 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65cfbe-3e52-485f-8e58-bb5d725bda09 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:42.653732 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65cfbe-3e52-485f-8e58-bb5d725bda09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:42.654040 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65cfbe-3e52-485f-8e58-bb5d725bda09 admin admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:42.654681 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d65cfbe-3e52-485f-8e58-bb5d725bda09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:21:42.654987 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d65cfbe-3e52-485f-8e58-bb5d725bda09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:42.656832 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d65cfbe-3e52-485f-8e58-bb5d725bda09 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:42.669188 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1314/2617] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:21:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:21:42.673297 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae975bf3-3e9f-47ca-b25f-c462ec7cb349 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:42.682659 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae975bf3-3e9f-47ca-b25f-c462ec7cb349 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:42.687375 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2470fdc9-62c9-4835-ac1a-6f5351b3f36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:42.687966 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2470fdc9-62c9-4835-ac1a-6f5351b3f36e admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:42.688289 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2470fdc9-62c9-4835-ac1a-6f5351b3f36e admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:42.688565 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2470fdc9-62c9-4835-ac1a-6f5351b3f36e admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:42.689187 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2470fdc9-62c9-4835-ac1a-6f5351b3f36e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:42.691091 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2470fdc9-62c9-4835-ac1a-6f5351b3f36e admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:42.719349 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1304/2618] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:21:42 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:21:42.723539 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d65cfbe-3e52-485f-8e58-bb5d725bda09 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:42.733769 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d65cfbe-3e52-485f-8e58-bb5d725bda09 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:42.798342 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f72a80-48a0-4d03-9068-530c76f7139d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:42.798965 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f72a80-48a0-4d03-9068-530c76f7139d admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:42.799224 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f72a80-48a0-4d03-9068-530c76f7139d admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:42.799502 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f72a80-48a0-4d03-9068-530c76f7139d admin admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:42.800307 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f72a80-48a0-4d03-9068-530c76f7139d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:42.802236 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f72a80-48a0-4d03-9068-530c76f7139d admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:42.824719 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1315/2619] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:21:42 2022] POST /identity/v3/users => generated 283 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 01:21:42.828543 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2470fdc9-62c9-4835-ac1a-6f5351b3f36e admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:42.839466 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2470fdc9-62c9-4835-ac1a-6f5351b3f36e admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:42.844265 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8760eb5-b74c-4ffb-afd0-7954c793c768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:42.844869 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8760eb5-b74c-4ffb-afd0-7954c793c768 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:42.845151 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8760eb5-b74c-4ffb-afd0-7954c793c768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:42.845449 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8760eb5-b74c-4ffb-afd0-7954c793c768 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:42.846090 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8760eb5-b74c-4ffb-afd0-7954c793c768 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:42.847689 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8760eb5-b74c-4ffb-afd0-7954c793c768 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:42.858095 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1305/2620] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:21:42.862232 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f72a80-48a0-4d03-9068-530c76f7139d admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:42.873057 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f72a80-48a0-4d03-9068-530c76f7139d admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:42.878099 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79bd2f9-9bbd-4cfb-976a-a3fb6ca18b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:42.878705 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79bd2f9-9bbd-4cfb-976a-a3fb6ca18b18 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:42.878979 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79bd2f9-9bbd-4cfb-976a-a3fb6ca18b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:42.879260 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79bd2f9-9bbd-4cfb-976a-a3fb6ca18b18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:42.879901 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79bd2f9-9bbd-4cfb-976a-a3fb6ca18b18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:42.881582 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79bd2f9-9bbd-4cfb-976a-a3fb6ca18b18 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:42.903197 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1316/2621] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:21:42.905084 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8760eb5-b74c-4ffb-afd0-7954c793c768 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:42.916561 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8760eb5-b74c-4ffb-afd0-7954c793c768 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:42.921334 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5ce868-a466-4ab0-bf27-8b1f4da5e9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:42.922000 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ce868-a466-4ab0-bf27-8b1f4da5e9d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:42.922282 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ce868-a466-4ab0-bf27-8b1f4da5e9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:42.922557 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ce868-a466-4ab0-bf27-8b1f4da5e9d2 admin admin] PATH_INFO: `/v3/projects/29172a5160f7415298950f9a8df90819/users/10310b867bf04a488291529f48155149/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:42.938196 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5ce868-a466-4ab0-bf27-8b1f4da5e9d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29172a5160f7415298950f9a8df90819, user_id=10310b867bf04a488291529f48155149, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:42.942150 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5ce868-a466-4ab0-bf27-8b1f4da5e9d2 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:42.956340 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1306/2622] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:21:42 2022] PUT /identity/v3/projects/29172a5160f7415298950f9a8df90819/users/10310b867bf04a488291529f48155149/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:21:42.970622 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91a27a-234d-4955-9e30-3dd7e002652c None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:42.970907 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91a27a-234d-4955-9e30-3dd7e002652c None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:42.971183 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91a27a-234d-4955-9e30-3dd7e002652c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:42.999035 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-df91a27a-234d-4955-9e30-3dd7e002652c None None] MFA Rules not processed for user `10310b867bf04a488291529f48155149`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:21:43.000274 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-df91a27a-234d-4955-9e30-3dd7e002652c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:43.059916 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:21:43.066299 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1317/2623] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:21:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:21:43.078403 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5ce868-a466-4ab0-bf27-8b1f4da5e9d2 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:43.080154 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-9d5ce868-a466-4ab0-bf27-8b1f4da5e9d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:43.136727 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5ce868-a466-4ab0-bf27-8b1f4da5e9d2 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:43.140487 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599623b6-61ef-480f-bf82-35b73add9d99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:43.141086 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599623b6-61ef-480f-bf82-35b73add9d99 service cinder] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:43.141368 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599623b6-61ef-480f-bf82-35b73add9d99 service cinder] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:43.141642 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599623b6-61ef-480f-bf82-35b73add9d99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:43.150212 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599623b6-61ef-480f-bf82-35b73add9d99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:43.151647 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599623b6-61ef-480f-bf82-35b73add9d99 service cinder] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:43.160320 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1307/2624] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:21:43 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:21:50.105200 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df91a27a-234d-4955-9e30-3dd7e002652c None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:50.106739 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-df91a27a-234d-4955-9e30-3dd7e002652c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:50.310025 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df91a27a-234d-4955-9e30-3dd7e002652c None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:50.314444 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aca263-fe22-4e3b-bb3e-aa5b59897a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:50.315068 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aca263-fe22-4e3b-bb3e-aa5b59897a04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:50.315342 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aca263-fe22-4e3b-bb3e-aa5b59897a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:50.315611 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aca263-fe22-4e3b-bb3e-aa5b59897a04 admin admin] PATH_INFO: `/v3/users/49a3d57540ad4814b196a655b306eb2d` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:50.316737 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aca263-fe22-4e3b-bb3e-aa5b59897a04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49a3d57540ad4814b196a655b306eb2d)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:50.318662 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aca263-fe22-4e3b-bb3e-aa5b59897a04 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:50.395434 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-a5aca263-fe22-4e3b-bb3e-aa5b59897a04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49a3d57540ad4814b196a655b306eb2d'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:50.411710 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-a5aca263-fe22-4e3b-bb3e-aa5b59897a04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49a3d57540ad4814b196a655b306eb2d'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:50.421873 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-a5aca263-fe22-4e3b-bb3e-aa5b59897a04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49a3d57540ad4814b196a655b306eb2d'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:50.440955 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-a5aca263-fe22-4e3b-bb3e-aa5b59897a04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49a3d57540ad4814b196a655b306eb2d'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:50.441756 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-a5aca263-fe22-4e3b-bb3e-aa5b59897a04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49a3d57540ad4814b196a655b306eb2d'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:50.450240 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1318/2625] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:21:50 2022] DELETE /identity/v3/users/49a3d57540ad4814b196a655b306eb2d => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:21:50.454517 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f470e2a8-43b7-4ad8-b4fa-cd78e5e3ae9b None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:50.455055 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f470e2a8-43b7-4ad8-b4fa-cd78e5e3ae9b None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:50.455158 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f470e2a8-43b7-4ad8-b4fa-cd78e5e3ae9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:50.477932 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-f470e2a8-43b7-4ad8-b4fa-cd78e5e3ae9b None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:21:50.479169 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-f470e2a8-43b7-4ad8-b4fa-cd78e5e3ae9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:50.540041 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1308/2626] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:21:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:21:50.549691 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aca263-fe22-4e3b-bb3e-aa5b59897a04 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:50.551177 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-a5aca263-fe22-4e3b-bb3e-aa5b59897a04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:50.611207 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aca263-fe22-4e3b-bb3e-aa5b59897a04 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:50.615111 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dafbff9-b94a-466a-9c60-dee2b3ed7b46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:50.615820 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dafbff9-b94a-466a-9c60-dee2b3ed7b46 service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:50.616183 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dafbff9-b94a-466a-9c60-dee2b3ed7b46 service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:50.616464 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dafbff9-b94a-466a-9c60-dee2b3ed7b46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:50.626886 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dafbff9-b94a-466a-9c60-dee2b3ed7b46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:50.628299 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dafbff9-b94a-466a-9c60-dee2b3ed7b46 service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:50.644520 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1319/2627] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:21:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:21:50.927584 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f470e2a8-43b7-4ad8-b4fa-cd78e5e3ae9b None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:50.929036 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-f470e2a8-43b7-4ad8-b4fa-cd78e5e3ae9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:50.941953 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f470e2a8-43b7-4ad8-b4fa-cd78e5e3ae9b None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:50.946987 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c9df83-4ab3-4c65-b26e-ad4886f80837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:50.947667 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9df83-4ab3-4c65-b26e-ad4886f80837 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:50.947913 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9df83-4ab3-4c65-b26e-ad4886f80837 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:50.948593 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9df83-4ab3-4c65-b26e-ad4886f80837 admin admin] PATH_INFO: `/v3/projects/89fc5b93a67d4b97be3cbcb8cdd7e3c3` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:50.949447 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c9df83-4ab3-4c65-b26e-ad4886f80837 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89fc5b93a67d4b97be3cbcb8cdd7e3c3)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:50.951401 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c9df83-4ab3-4c65-b26e-ad4886f80837 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:50.958999 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:21:51.002543 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-81c9df83-4ab3-4c65-b26e-ad4886f80837 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89fc5b93a67d4b97be3cbcb8cdd7e3c3', 'request_id': 'req-81c9df83-4ab3-4c65-b26e-ad4886f80837'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:51.010315 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-81c9df83-4ab3-4c65-b26e-ad4886f80837 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89fc5b93a67d4b97be3cbcb8cdd7e3c3', 'request_id': 'req-81c9df83-4ab3-4c65-b26e-ad4886f80837'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:51.016502 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-81c9df83-4ab3-4c65-b26e-ad4886f80837 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89fc5b93a67d4b97be3cbcb8cdd7e3c3', 'request_id': 'req-81c9df83-4ab3-4c65-b26e-ad4886f80837'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:51.027201 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-81c9df83-4ab3-4c65-b26e-ad4886f80837 admin admin] The token cache is being invalidate because project 89fc5b93a67d4b97be3cbcb8cdd7e3c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81661) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:21:51.027493 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-81c9df83-4ab3-4c65-b26e-ad4886f80837 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:51.028122 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-81c9df83-4ab3-4c65-b26e-ad4886f80837 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:51.029636 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1309/2628] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:21:50 2022] DELETE /identity/v3/projects/89fc5b93a67d4b97be3cbcb8cdd7e3c3 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:21:51.034414 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa807258-3119-4469-9ef0-d653ac4dd04c None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:51.034716 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa807258-3119-4469-9ef0-d653ac4dd04c None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:51.034972 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa807258-3119-4469-9ef0-d653ac4dd04c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:51.055209 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-fa807258-3119-4469-9ef0-d653ac4dd04c None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:21:51.056540 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-fa807258-3119-4469-9ef0-d653ac4dd04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:51.069113 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:21:51.116635 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1320/2629] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:21:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:21:51.125020 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e734eb5-305e-49b3-af6b-833ca9fb45b7 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:51.125304 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e734eb5-305e-49b3-af6b-833ca9fb45b7 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:51.125580 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e734eb5-305e-49b3-af6b-833ca9fb45b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:51.145360 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-6e734eb5-305e-49b3-af6b-833ca9fb45b7 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:21:51.146571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-6e734eb5-305e-49b3-af6b-833ca9fb45b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:51.155556 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1310/2630] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:21:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:21:51.159695 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa807258-3119-4469-9ef0-d653ac4dd04c None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:51.170032 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa807258-3119-4469-9ef0-d653ac4dd04c None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:51.174661 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a3c69e-a545-4e1a-8214-930a61cb2a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:51.175247 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a3c69e-a545-4e1a-8214-930a61cb2a3c admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:51.175521 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a3c69e-a545-4e1a-8214-930a61cb2a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:51.175786 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a3c69e-a545-4e1a-8214-930a61cb2a3c admin admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:51.176455 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a3c69e-a545-4e1a-8214-930a61cb2a3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:21:51.176799 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a3c69e-a545-4e1a-8214-930a61cb2a3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:51.178361 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a3c69e-a545-4e1a-8214-930a61cb2a3c admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:51.189870 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1321/2631] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:21:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:21:51.194267 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e734eb5-305e-49b3-af6b-833ca9fb45b7 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:51.203794 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e734eb5-305e-49b3-af6b-833ca9fb45b7 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:51.208509 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3b484d-6804-4e92-8089-43a47f2a3e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:51.209101 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b484d-6804-4e92-8089-43a47f2a3e5e admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:51.209377 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b484d-6804-4e92-8089-43a47f2a3e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:51.209648 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b484d-6804-4e92-8089-43a47f2a3e5e admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:51.210288 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3b484d-6804-4e92-8089-43a47f2a3e5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:51.212826 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3b484d-6804-4e92-8089-43a47f2a3e5e admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:51.239747 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1311/2632] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:21:51 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:21:51.243819 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a3c69e-a545-4e1a-8214-930a61cb2a3c admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:51.253922 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a3c69e-a545-4e1a-8214-930a61cb2a3c admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:51.315522 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c851006-46cf-48a1-89d9-6f4d040c950c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:51.316210 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c851006-46cf-48a1-89d9-6f4d040c950c admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:51.316493 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c851006-46cf-48a1-89d9-6f4d040c950c admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:51.316765 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c851006-46cf-48a1-89d9-6f4d040c950c admin admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:51.317406 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c851006-46cf-48a1-89d9-6f4d040c950c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:51.319495 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c851006-46cf-48a1-89d9-6f4d040c950c admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:51.341492 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1322/2633] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:21:51 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:21:51.345561 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3b484d-6804-4e92-8089-43a47f2a3e5e admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:51.355933 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3b484d-6804-4e92-8089-43a47f2a3e5e admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:51.360642 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8df9f4b-13af-4839-9f71-120bf01b7b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:51.361234 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df9f4b-13af-4839-9f71-120bf01b7b6b admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:51.361520 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df9f4b-13af-4839-9f71-120bf01b7b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:51.361793 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df9f4b-13af-4839-9f71-120bf01b7b6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:51.362432 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8df9f4b-13af-4839-9f71-120bf01b7b6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:51.364038 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8df9f4b-13af-4839-9f71-120bf01b7b6b admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:51.374310 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1312/2634] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:21:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:21:51.378096 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c851006-46cf-48a1-89d9-6f4d040c950c admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:51.388852 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c851006-46cf-48a1-89d9-6f4d040c950c admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:51.393511 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5df522f-2edc-4f2b-bce9-154f6c14f962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:51.394105 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df522f-2edc-4f2b-bce9-154f6c14f962 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:51.394382 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df522f-2edc-4f2b-bce9-154f6c14f962 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:51.394652 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df522f-2edc-4f2b-bce9-154f6c14f962 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:51.395289 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5df522f-2edc-4f2b-bce9-154f6c14f962 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:51.397094 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5df522f-2edc-4f2b-bce9-154f6c14f962 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:51.407101 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1323/2635] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:21:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:21:51.411146 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8df9f4b-13af-4839-9f71-120bf01b7b6b admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:51.421587 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8df9f4b-13af-4839-9f71-120bf01b7b6b admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:51.426291 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cfc1ab-c692-49ea-af72-5df348433050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:51.426965 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cfc1ab-c692-49ea-af72-5df348433050 admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:51.427248 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cfc1ab-c692-49ea-af72-5df348433050 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:51.427523 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cfc1ab-c692-49ea-af72-5df348433050 admin admin] PATH_INFO: `/v3/projects/5f8e546223a549a1a9997c146fbf6d4c/users/a90a6c4d42ca4f14b9baf123eb4d6fa2/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:51.443316 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cfc1ab-c692-49ea-af72-5df348433050 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f8e546223a549a1a9997c146fbf6d4c, user_id=a90a6c4d42ca4f14b9baf123eb4d6fa2, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:51.446687 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cfc1ab-c692-49ea-af72-5df348433050 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:51.460750 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1313/2636] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:21:51 2022] PUT /identity/v3/projects/5f8e546223a549a1a9997c146fbf6d4c/users/a90a6c4d42ca4f14b9baf123eb4d6fa2/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:21:51.471264 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2261ec9a-1418-4088-8d72-52a5e1a71401 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:51.471474 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2261ec9a-1418-4088-8d72-52a5e1a71401 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:51.472338 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2261ec9a-1418-4088-8d72-52a5e1a71401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:51.502411 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-2261ec9a-1418-4088-8d72-52a5e1a71401 None None] MFA Rules not processed for user `a90a6c4d42ca4f14b9baf123eb4d6fa2`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:21:51.503605 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-2261ec9a-1418-4088-8d72-52a5e1a71401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:51.582446 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1324/2637] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:21:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:21:51.590672 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cfc1ab-c692-49ea-af72-5df348433050 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:51.592411 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-96cfc1ab-c692-49ea-af72-5df348433050 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:51.672820 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cfc1ab-c692-49ea-af72-5df348433050 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:51.676712 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6e0307-f2f8-4f46-a3c4-b3a6ecbe1135 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:51.677474 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e0307-f2f8-4f46-a3c4-b3a6ecbe1135 service cinder] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:51.677784 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e0307-f2f8-4f46-a3c4-b3a6ecbe1135 service cinder] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:51.678077 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e0307-f2f8-4f46-a3c4-b3a6ecbe1135 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:51.687148 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6e0307-f2f8-4f46-a3c4-b3a6ecbe1135 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:51.688623 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6e0307-f2f8-4f46-a3c4-b3a6ecbe1135 service cinder] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:51.698301 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1314/2638] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:21:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:21:53.751415 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2261ec9a-1418-4088-8d72-52a5e1a71401 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:53.753024 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-2261ec9a-1418-4088-8d72-52a5e1a71401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:53.772939 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:21:53.822436 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2261ec9a-1418-4088-8d72-52a5e1a71401 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:53.827096 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6262710d-3a3d-42fc-9964-f927811ff5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:53.827721 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6262710d-3a3d-42fc-9964-f927811ff5c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:53.827999 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6262710d-3a3d-42fc-9964-f927811ff5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:53.828373 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6262710d-3a3d-42fc-9964-f927811ff5c8 admin admin] PATH_INFO: `/v3/users/10310b867bf04a488291529f48155149` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:53.829681 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6262710d-3a3d-42fc-9964-f927811ff5c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10310b867bf04a488291529f48155149)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:53.831591 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6262710d-3a3d-42fc-9964-f927811ff5c8 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:53.888098 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-6262710d-3a3d-42fc-9964-f927811ff5c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10310b867bf04a488291529f48155149'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:53.902995 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-6262710d-3a3d-42fc-9964-f927811ff5c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10310b867bf04a488291529f48155149'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:53.910815 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-6262710d-3a3d-42fc-9964-f927811ff5c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10310b867bf04a488291529f48155149'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:53.921438 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-6262710d-3a3d-42fc-9964-f927811ff5c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10310b867bf04a488291529f48155149'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:53.922268 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-6262710d-3a3d-42fc-9964-f927811ff5c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10310b867bf04a488291529f48155149'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:53.931687 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1325/2639] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:21:53 2022] DELETE /identity/v3/users/10310b867bf04a488291529f48155149 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:21:53.937833 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912dbde-6e9a-4b8d-b2ae-d6055e656d12 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:53.938810 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912dbde-6e9a-4b8d-b2ae-d6055e656d12 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:53.938810 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912dbde-6e9a-4b8d-b2ae-d6055e656d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:53.972153 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-4912dbde-6e9a-4b8d-b2ae-d6055e656d12 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:21:53.974408 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-4912dbde-6e9a-4b8d-b2ae-d6055e656d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:53.995045 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:21:54.073179 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1315/2640] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:21:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:21:54.082704 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6262710d-3a3d-42fc-9964-f927811ff5c8 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:54.084352 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-6262710d-3a3d-42fc-9964-f927811ff5c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:54.146873 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6262710d-3a3d-42fc-9964-f927811ff5c8 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:54.151340 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653ed878-e564-452b-87ff-36aca41d8197 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:54.151943 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ed878-e564-452b-87ff-36aca41d8197 service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:54.152244 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ed878-e564-452b-87ff-36aca41d8197 service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:54.152519 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ed878-e564-452b-87ff-36aca41d8197 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:54.162035 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653ed878-e564-452b-87ff-36aca41d8197 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:54.163481 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653ed878-e564-452b-87ff-36aca41d8197 service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:54.174778 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1326/2641] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:21:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:21:54.474307 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4912dbde-6e9a-4b8d-b2ae-d6055e656d12 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:54.475731 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-4912dbde-6e9a-4b8d-b2ae-d6055e656d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:54.489600 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4912dbde-6e9a-4b8d-b2ae-d6055e656d12 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:54.495076 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fec33c-31ad-4f18-96d1-7344d064e653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:54.495778 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fec33c-31ad-4f18-96d1-7344d064e653 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:54.496057 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fec33c-31ad-4f18-96d1-7344d064e653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:54.496419 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fec33c-31ad-4f18-96d1-7344d064e653 admin admin] PATH_INFO: `/v3/projects/29172a5160f7415298950f9a8df90819` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:54.497562 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fec33c-31ad-4f18-96d1-7344d064e653 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29172a5160f7415298950f9a8df90819)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:54.499546 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fec33c-31ad-4f18-96d1-7344d064e653 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:54.548337 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-09fec33c-31ad-4f18-96d1-7344d064e653 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29172a5160f7415298950f9a8df90819', 'request_id': 'req-09fec33c-31ad-4f18-96d1-7344d064e653'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:54.556842 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-09fec33c-31ad-4f18-96d1-7344d064e653 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29172a5160f7415298950f9a8df90819', 'request_id': 'req-09fec33c-31ad-4f18-96d1-7344d064e653'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:54.563445 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-09fec33c-31ad-4f18-96d1-7344d064e653 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29172a5160f7415298950f9a8df90819', 'request_id': 'req-09fec33c-31ad-4f18-96d1-7344d064e653'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:54.575060 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-09fec33c-31ad-4f18-96d1-7344d064e653 admin admin] The token cache is being invalidate because project 29172a5160f7415298950f9a8df90819 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81661) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:21:54.575347 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-09fec33c-31ad-4f18-96d1-7344d064e653 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:54.576003 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-09fec33c-31ad-4f18-96d1-7344d064e653 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:21:54.577491 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1316/2642] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:21:54 2022] DELETE /identity/v3/projects/29172a5160f7415298950f9a8df90819 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:21:54.582267 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105d4d1-7ee3-461b-a18c-9dd5dd4c31d4 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:54.582548 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105d4d1-7ee3-461b-a18c-9dd5dd4c31d4 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:54.582815 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105d4d1-7ee3-461b-a18c-9dd5dd4c31d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:54.605953 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-1105d4d1-7ee3-461b-a18c-9dd5dd4c31d4 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:21:54.607161 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-1105d4d1-7ee3-461b-a18c-9dd5dd4c31d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:54.648756 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:21:54.677957 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1327/2643] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:21:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:21:54.686070 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d196ba7-49a8-4de0-8e8f-30cca0d1c2af None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:54.686355 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d196ba7-49a8-4de0-8e8f-30cca0d1c2af None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:54.686629 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d196ba7-49a8-4de0-8e8f-30cca0d1c2af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:54.707959 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-3d196ba7-49a8-4de0-8e8f-30cca0d1c2af None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:21:54.709561 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-3d196ba7-49a8-4de0-8e8f-30cca0d1c2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:54.719034 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1317/2644] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:21:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:21:54.723649 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1105d4d1-7ee3-461b-a18c-9dd5dd4c31d4 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:54.733862 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1105d4d1-7ee3-461b-a18c-9dd5dd4c31d4 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:54.738575 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bfea77-6c40-41d7-a86c-e6bfd9e800fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:54.739169 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bfea77-6c40-41d7-a86c-e6bfd9e800fc admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:54.739442 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bfea77-6c40-41d7-a86c-e6bfd9e800fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:54.739856 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bfea77-6c40-41d7-a86c-e6bfd9e800fc admin admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:54.740569 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bfea77-6c40-41d7-a86c-e6bfd9e800fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:21:54.740869 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bfea77-6c40-41d7-a86c-e6bfd9e800fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:54.742817 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bfea77-6c40-41d7-a86c-e6bfd9e800fc admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:54.756208 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1328/2645] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:21:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:21:54.760326 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d196ba7-49a8-4de0-8e8f-30cca0d1c2af None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:54.771637 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d196ba7-49a8-4de0-8e8f-30cca0d1c2af None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:54.778451 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46521ee-78d2-4212-b4a0-c929ccaac16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:54.779152 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46521ee-78d2-4212-b4a0-c929ccaac16c admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:54.779442 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46521ee-78d2-4212-b4a0-c929ccaac16c admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:54.779719 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46521ee-78d2-4212-b4a0-c929ccaac16c admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:54.780405 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46521ee-78d2-4212-b4a0-c929ccaac16c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:54.782832 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46521ee-78d2-4212-b4a0-c929ccaac16c admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:54.813707 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1318/2646] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:21:54 2022] POST /identity/v3/projects => generated 363 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:21:54.817746 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bfea77-6c40-41d7-a86c-e6bfd9e800fc admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:54.828298 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bfea77-6c40-41d7-a86c-e6bfd9e800fc admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:54.900935 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1755072b-f6d5-4bdb-b461-496e15aa8d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:54.901614 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1755072b-f6d5-4bdb-b461-496e15aa8d67 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:54.901964 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1755072b-f6d5-4bdb-b461-496e15aa8d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:54.902344 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1755072b-f6d5-4bdb-b461-496e15aa8d67 admin admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:54.903227 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1755072b-f6d5-4bdb-b461-496e15aa8d67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:54.905252 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1755072b-f6d5-4bdb-b461-496e15aa8d67 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:54.928931 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1329/2647] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:21:54 2022] POST /identity/v3/users => generated 282 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:21:54.933883 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46521ee-78d2-4212-b4a0-c929ccaac16c admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:54.948413 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46521ee-78d2-4212-b4a0-c929ccaac16c admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:54.953149 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46aee8b-257d-47d3-af0f-dcbd372a7052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:54.953756 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46aee8b-257d-47d3-af0f-dcbd372a7052 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:54.954032 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46aee8b-257d-47d3-af0f-dcbd372a7052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:54.954302 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46aee8b-257d-47d3-af0f-dcbd372a7052 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:54.954941 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46aee8b-257d-47d3-af0f-dcbd372a7052 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:54.956650 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46aee8b-257d-47d3-af0f-dcbd372a7052 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:54.974591 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1319/2648] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:21:54 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:21:54.979948 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1755072b-f6d5-4bdb-b461-496e15aa8d67 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:54.993041 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1755072b-f6d5-4bdb-b461-496e15aa8d67 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:54.998535 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c28e696-1bcb-4a13-bd24-98ef11089553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:54.999141 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28e696-1bcb-4a13-bd24-98ef11089553 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:54.999419 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28e696-1bcb-4a13-bd24-98ef11089553 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:54.999696 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28e696-1bcb-4a13-bd24-98ef11089553 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:55.000374 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c28e696-1bcb-4a13-bd24-98ef11089553 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:55.002119 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c28e696-1bcb-4a13-bd24-98ef11089553 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:55.014914 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1330/2649] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:21:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:21:55.019859 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46aee8b-257d-47d3-af0f-dcbd372a7052 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:55.030621 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46aee8b-257d-47d3-af0f-dcbd372a7052 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:55.035350 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a74b00a-f366-4b1e-bbd8-d9e5fe135933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:55.036000 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a74b00a-f366-4b1e-bbd8-d9e5fe135933 admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:55.036318 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a74b00a-f366-4b1e-bbd8-d9e5fe135933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:55.036597 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a74b00a-f366-4b1e-bbd8-d9e5fe135933 admin admin] PATH_INFO: `/v3/projects/c29767f090d84948924ae79b27aa7d6b/users/5cafdaf69e304c7389fec32e0698531e/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:55.051371 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a74b00a-f366-4b1e-bbd8-d9e5fe135933 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c29767f090d84948924ae79b27aa7d6b, user_id=5cafdaf69e304c7389fec32e0698531e, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:55.054092 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a74b00a-f366-4b1e-bbd8-d9e5fe135933 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:55.067954 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1320/2650] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:21:55 2022] PUT /identity/v3/projects/c29767f090d84948924ae79b27aa7d6b/users/5cafdaf69e304c7389fec32e0698531e/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:21:55.079774 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647dfa9-4e9d-4c1e-801c-6a0a6a2d24d5 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:55.080102 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647dfa9-4e9d-4c1e-801c-6a0a6a2d24d5 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:55.080580 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647dfa9-4e9d-4c1e-801c-6a0a6a2d24d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:55.115186 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-6647dfa9-4e9d-4c1e-801c-6a0a6a2d24d5 None None] MFA Rules not processed for user `5cafdaf69e304c7389fec32e0698531e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:21:55.117418 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-6647dfa9-4e9d-4c1e-801c-6a0a6a2d24d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:55.199934 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1331/2651] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:21:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:21:55.206457 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a74b00a-f366-4b1e-bbd8-d9e5fe135933 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:55.243809 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:21:55.277216 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a74b00a-f366-4b1e-bbd8-d9e5fe135933 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:55.281954 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901886ba-6c0d-4232-82ad-4b8158874a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:55.282562 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901886ba-6c0d-4232-82ad-4b8158874a42 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:55.282845 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901886ba-6c0d-4232-82ad-4b8158874a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:55.283113 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901886ba-6c0d-4232-82ad-4b8158874a42 admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:55.283758 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901886ba-6c0d-4232-82ad-4b8158874a42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:55.285748 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901886ba-6c0d-4232-82ad-4b8158874a42 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:55.317434 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1321/2652] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:21:55 2022] POST /identity/v3/projects => generated 365 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 01:21:55.319396 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6647dfa9-4e9d-4c1e-801c-6a0a6a2d24d5 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:55.331947 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6647dfa9-4e9d-4c1e-801c-6a0a6a2d24d5 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:55.375901 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:21:55.405271 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34122722-2c94-4ea4-a7b3-441a67312c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:55.405883 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34122722-2c94-4ea4-a7b3-441a67312c91 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:55.406158 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34122722-2c94-4ea4-a7b3-441a67312c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:55.406420 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34122722-2c94-4ea4-a7b3-441a67312c91 admin admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:55.407219 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34122722-2c94-4ea4-a7b3-441a67312c91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:55.409222 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34122722-2c94-4ea4-a7b3-441a67312c91 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:55.432655 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1332/2653] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:21:55 2022] POST /identity/v3/users => generated 289 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 01:21:55.436384 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901886ba-6c0d-4232-82ad-4b8158874a42 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:55.446986 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901886ba-6c0d-4232-82ad-4b8158874a42 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:55.451654 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcc84b9-8061-426e-b490-cab8e6f5a055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:55.452292 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc84b9-8061-426e-b490-cab8e6f5a055 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:55.452574 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc84b9-8061-426e-b490-cab8e6f5a055 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:55.452849 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc84b9-8061-426e-b490-cab8e6f5a055 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:55.453491 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcc84b9-8061-426e-b490-cab8e6f5a055 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:55.455104 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcc84b9-8061-426e-b490-cab8e6f5a055 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:55.465858 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1322/2654] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:21:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:21:55.470006 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34122722-2c94-4ea4-a7b3-441a67312c91 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:55.480706 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34122722-2c94-4ea4-a7b3-441a67312c91 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:55.485411 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020a696-5128-414d-94d5-73e76fc1eda8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:55.486027 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020a696-5128-414d-94d5-73e76fc1eda8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:55.486301 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020a696-5128-414d-94d5-73e76fc1eda8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:55.486570 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020a696-5128-414d-94d5-73e76fc1eda8 admin admin] PATH_INFO: `/v3/projects/f81400a930704b158c9cc09545e58aea/users/3fc092790acd485986651b63912399be/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:55.502391 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a020a696-5128-414d-94d5-73e76fc1eda8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f81400a930704b158c9cc09545e58aea, user_id=3fc092790acd485986651b63912399be, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:55.505063 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a020a696-5128-414d-94d5-73e76fc1eda8 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:55.519225 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1333/2655] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:21:55 2022] PUT /identity/v3/projects/f81400a930704b158c9cc09545e58aea/users/3fc092790acd485986651b63912399be/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:21:55.523001 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcc84b9-8061-426e-b490-cab8e6f5a055 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:55.533227 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcc84b9-8061-426e-b490-cab8e6f5a055 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:55.592618 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b819d3-c499-49d0-9eea-1bbb799e4705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:55.593228 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b819d3-c499-49d0-9eea-1bbb799e4705 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:55.593546 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b819d3-c499-49d0-9eea-1bbb799e4705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:55.593846 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b819d3-c499-49d0-9eea-1bbb799e4705 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:55.594508 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b819d3-c499-49d0-9eea-1bbb799e4705 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:55.596115 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b819d3-c499-49d0-9eea-1bbb799e4705 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:55.610405 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1323/2656] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:21:55 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:21:55.611537 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020a696-5128-414d-94d5-73e76fc1eda8 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:55.623189 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020a696-5128-414d-94d5-73e76fc1eda8 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:55.628022 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f4b37e-9f9f-4202-84bc-21c48c62a831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:55.628787 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4b37e-9f9f-4202-84bc-21c48c62a831 admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:55.629064 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4b37e-9f9f-4202-84bc-21c48c62a831 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:55.629340 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4b37e-9f9f-4202-84bc-21c48c62a831 admin admin] PATH_INFO: `/v3/projects/f81400a930704b158c9cc09545e58aea/users/3fc092790acd485986651b63912399be/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:55.631686 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f4b37e-9f9f-4202-84bc-21c48c62a831 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f81400a930704b158c9cc09545e58aea, user_id=3fc092790acd485986651b63912399be, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:55.634911 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f4b37e-9f9f-4202-84bc-21c48c62a831 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:55.646423 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1334/2657] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:21:55 2022] PUT /identity/v3/projects/f81400a930704b158c9cc09545e58aea/users/3fc092790acd485986651b63912399be/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:21:55.659869 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa2ccb-55c6-4e56-8c51-635c03fd9c86 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:55.660179 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa2ccb-55c6-4e56-8c51-635c03fd9c86 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:55.660463 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa2ccb-55c6-4e56-8c51-635c03fd9c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:55.688952 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-99aa2ccb-55c6-4e56-8c51-635c03fd9c86 None None] MFA Rules not processed for user `3fc092790acd485986651b63912399be`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:21:55.690171 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-99aa2ccb-55c6-4e56-8c51-635c03fd9c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:55.765464 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1324/2658] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:21:55 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:21:55.778193 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f4b37e-9f9f-4202-84bc-21c48c62a831 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:55.779963 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-c0f4b37e-9f9f-4202-84bc-21c48c62a831 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:55.841686 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f4b37e-9f9f-4202-84bc-21c48c62a831 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:55.845357 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b71224-2f29-4e2c-960d-a1093e1bb144 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:55.845956 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b71224-2f29-4e2c-960d-a1093e1bb144 service cinder] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:55.846234 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b71224-2f29-4e2c-960d-a1093e1bb144 service cinder] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:55.846501 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b71224-2f29-4e2c-960d-a1093e1bb144 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:55.856384 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b71224-2f29-4e2c-960d-a1093e1bb144 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:55.857808 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b71224-2f29-4e2c-960d-a1093e1bb144 service cinder] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:55.869216 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1335/2659] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:21:55 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:21:55.954254 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aa2ccb-55c6-4e56-8c51-635c03fd9c86 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:55.964508 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aa2ccb-55c6-4e56-8c51-635c03fd9c86 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:55.968209 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8eeb06-11fd-4d17-8795-cbd3f43be741 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:55.968812 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8eeb06-11fd-4d17-8795-cbd3f43be741 service cinder] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:55.969095 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8eeb06-11fd-4d17-8795-cbd3f43be741 service cinder] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:55.969367 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8eeb06-11fd-4d17-8795-cbd3f43be741 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:55.998338 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:21:56.032024 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8eeb06-11fd-4d17-8795-cbd3f43be741 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:56.039742 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8eeb06-11fd-4d17-8795-cbd3f43be741 service cinder] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:56.045381 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1325/2660] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:21:55 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:21:59.532590 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b71224-2f29-4e2c-960d-a1093e1bb144 service cinder] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:21:59.534001 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-11b71224-2f29-4e2c-960d-a1093e1bb144 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:59.594605 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b71224-2f29-4e2c-960d-a1093e1bb144 service cinder] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:21:59.598234 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadee8d2-efb0-424b-b5be-ae5b0e6098b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:21:59.598972 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadee8d2-efb0-424b-b5be-ae5b0e6098b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:21:59.599246 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadee8d2-efb0-424b-b5be-ae5b0e6098b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:21:59.599514 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadee8d2-efb0-424b-b5be-ae5b0e6098b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:21:59.602103 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-aadee8d2-efb0-424b-b5be-ae5b0e6098b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:21:59.648945 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:21:59.670506 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadee8d2-efb0-424b-b5be-ae5b0e6098b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:21:59.671934 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadee8d2-efb0-424b-b5be-ae5b0e6098b9 service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:21:59.681759 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1336/2661] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:21:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:22:02.516710 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8eeb06-11fd-4d17-8795-cbd3f43be741 service cinder] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:02.518165 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-1d8eeb06-11fd-4d17-8795-cbd3f43be741 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:02.600997 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadee8d2-efb0-424b-b5be-ae5b0e6098b9 service neutron] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:02.602448 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-aadee8d2-efb0-424b-b5be-ae5b0e6098b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:02.627238 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadee8d2-efb0-424b-b5be-ae5b0e6098b9 service neutron] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:02.642325 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b5252-dd99-4c88-84bc-35aff967be67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:02.643075 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b5252-dd99-4c88-84bc-35aff967be67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:02.643530 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b5252-dd99-4c88-84bc-35aff967be67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:02.643809 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b5252-dd99-4c88-84bc-35aff967be67 admin admin] PATH_INFO: `/v3/users/83fea9762aaf4766a23f1774ff1411fe` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:02.645187 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809b5252-dd99-4c88-84bc-35aff967be67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83fea9762aaf4766a23f1774ff1411fe)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:02.647500 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809b5252-dd99-4c88-84bc-35aff967be67 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:02.658283 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8eeb06-11fd-4d17-8795-cbd3f43be741 service cinder] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:02.664497 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1799816-3517-497c-8670-8c7afd0c0599 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '301d30bbd3de4cc69e96bb12729442c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:02.665116 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1799816-3517-497c-8670-8c7afd0c0599 service placement] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:02.665400 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1799816-3517-497c-8670-8c7afd0c0599 service placement] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:02.665675 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1799816-3517-497c-8670-8c7afd0c0599 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:02.667548 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-b1799816-3517-497c-8670-8c7afd0c0599 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:02.681313 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1799816-3517-497c-8670-8c7afd0c0599 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:02.683835 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1799816-3517-497c-8670-8c7afd0c0599 service placement] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:02.697421 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1326/2662] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:22:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:22:02.714471 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-809b5252-dd99-4c88-84bc-35aff967be67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83fea9762aaf4766a23f1774ff1411fe'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:02.728707 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-809b5252-dd99-4c88-84bc-35aff967be67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83fea9762aaf4766a23f1774ff1411fe'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:02.736428 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-809b5252-dd99-4c88-84bc-35aff967be67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83fea9762aaf4766a23f1774ff1411fe'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:02.745482 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-809b5252-dd99-4c88-84bc-35aff967be67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83fea9762aaf4766a23f1774ff1411fe'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:02.746110 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-809b5252-dd99-4c88-84bc-35aff967be67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83fea9762aaf4766a23f1774ff1411fe'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:02.761344 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1337/2663] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:22:02 2022] DELETE /identity/v3/users/83fea9762aaf4766a23f1774ff1411fe => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:02.941497 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1799816-3517-497c-8670-8c7afd0c0599 service placement] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:02.952259 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1799816-3517-497c-8670-8c7afd0c0599 service placement] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:03.011111 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed250ab2-be96-41fa-bd6b-b432db9eaf73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:03.011710 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed250ab2-be96-41fa-bd6b-b432db9eaf73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:03.011993 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed250ab2-be96-41fa-bd6b-b432db9eaf73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:03.012306 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed250ab2-be96-41fa-bd6b-b432db9eaf73 admin admin] PATH_INFO: `/v3/projects/e856bb81a717482cbea65029d521700d` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:03.013422 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed250ab2-be96-41fa-bd6b-b432db9eaf73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e856bb81a717482cbea65029d521700d)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:03.015342 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed250ab2-be96-41fa-bd6b-b432db9eaf73 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:03.061865 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-ed250ab2-be96-41fa-bd6b-b432db9eaf73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e856bb81a717482cbea65029d521700d', 'request_id': 'req-ed250ab2-be96-41fa-bd6b-b432db9eaf73'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:03.068927 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-ed250ab2-be96-41fa-bd6b-b432db9eaf73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e856bb81a717482cbea65029d521700d', 'request_id': 'req-ed250ab2-be96-41fa-bd6b-b432db9eaf73'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:03.071039 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:03.076855 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-ed250ab2-be96-41fa-bd6b-b432db9eaf73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e856bb81a717482cbea65029d521700d', 'request_id': 'req-ed250ab2-be96-41fa-bd6b-b432db9eaf73'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:03.088243 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-ed250ab2-be96-41fa-bd6b-b432db9eaf73 admin admin] The token cache is being invalidate because project e856bb81a717482cbea65029d521700d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81661) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:22:03.088557 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-ed250ab2-be96-41fa-bd6b-b432db9eaf73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:03.089227 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-ed250ab2-be96-41fa-bd6b-b432db9eaf73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:03.090766 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1327/2664] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:22:02 2022] DELETE /identity/v3/projects/e856bb81a717482cbea65029d521700d => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:03.095710 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189fd936-1868-412e-8280-bc44771e495d None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:03.096021 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189fd936-1868-412e-8280-bc44771e495d None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:03.096325 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189fd936-1868-412e-8280-bc44771e495d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:03.118550 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-189fd936-1868-412e-8280-bc44771e495d None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:03.119769 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-189fd936-1868-412e-8280-bc44771e495d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:03.183566 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1338/2665] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:22:03.192274 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c75c5-121e-43b9-94ea-75bdca58da50 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:03.192565 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c75c5-121e-43b9-94ea-75bdca58da50 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:03.192846 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c75c5-121e-43b9-94ea-75bdca58da50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:03.213207 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-b61c75c5-121e-43b9-94ea-75bdca58da50 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:03.214447 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-b61c75c5-121e-43b9-94ea-75bdca58da50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:03.222525 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1328/2666] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:22:03.227593 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189fd936-1868-412e-8280-bc44771e495d None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:03.237664 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189fd936-1868-412e-8280-bc44771e495d None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:03.242633 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d273a-abe4-4d6c-b16e-02557e2543bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:03.243226 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d273a-abe4-4d6c-b16e-02557e2543bf admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:03.243506 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d273a-abe4-4d6c-b16e-02557e2543bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:03.243779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d273a-abe4-4d6c-b16e-02557e2543bf admin admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:03.244457 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25d273a-abe4-4d6c-b16e-02557e2543bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:22:03.244761 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25d273a-abe4-4d6c-b16e-02557e2543bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:03.246354 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25d273a-abe4-4d6c-b16e-02557e2543bf admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:03.257872 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1339/2667] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:22:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:22:03.262393 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c75c5-121e-43b9-94ea-75bdca58da50 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:03.271635 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c75c5-121e-43b9-94ea-75bdca58da50 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:03.277122 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f24d9-37bc-4fb5-8d6f-381f5c258bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:03.277715 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f24d9-37bc-4fb5-8d6f-381f5c258bbb admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:03.277997 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f24d9-37bc-4fb5-8d6f-381f5c258bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:03.278274 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f24d9-37bc-4fb5-8d6f-381f5c258bbb admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:03.278901 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9f24d9-37bc-4fb5-8d6f-381f5c258bbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:03.280866 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9f24d9-37bc-4fb5-8d6f-381f5c258bbb admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:03.308157 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1329/2668] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:22:03 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:22:03.312381 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d273a-abe4-4d6c-b16e-02557e2543bf admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:03.322599 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d273a-abe4-4d6c-b16e-02557e2543bf admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:03.356464 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:03.381009 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f406bd-4095-4e65-babe-7d6ef85fd979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:03.381610 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f406bd-4095-4e65-babe-7d6ef85fd979 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:03.381888 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f406bd-4095-4e65-babe-7d6ef85fd979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:03.382190 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f406bd-4095-4e65-babe-7d6ef85fd979 admin admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:03.383000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f406bd-4095-4e65-babe-7d6ef85fd979 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:03.385220 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f406bd-4095-4e65-babe-7d6ef85fd979 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:03.407092 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1340/2669] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:22:03 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:22:03.411358 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f24d9-37bc-4fb5-8d6f-381f5c258bbb admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:03.422496 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f24d9-37bc-4fb5-8d6f-381f5c258bbb admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:03.427163 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e81ed08-e004-4d78-83b0-6b719d94ff05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:03.427760 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e81ed08-e004-4d78-83b0-6b719d94ff05 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:03.428040 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e81ed08-e004-4d78-83b0-6b719d94ff05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:03.428369 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e81ed08-e004-4d78-83b0-6b719d94ff05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:03.429077 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e81ed08-e004-4d78-83b0-6b719d94ff05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:03.430685 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e81ed08-e004-4d78-83b0-6b719d94ff05 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:03.440510 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1330/2670] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:22:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:22:03.444461 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f406bd-4095-4e65-babe-7d6ef85fd979 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:03.454303 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f406bd-4095-4e65-babe-7d6ef85fd979 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:03.458926 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d253a7-f502-4ccb-bb75-45bb88718534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:03.459511 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d253a7-f502-4ccb-bb75-45bb88718534 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:03.459792 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d253a7-f502-4ccb-bb75-45bb88718534 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:03.460068 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d253a7-f502-4ccb-bb75-45bb88718534 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:03.461401 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d253a7-f502-4ccb-bb75-45bb88718534 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:03.462982 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d253a7-f502-4ccb-bb75-45bb88718534 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:03.473374 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1341/2671] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:22:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:22:03.478017 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e81ed08-e004-4d78-83b0-6b719d94ff05 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:03.487878 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e81ed08-e004-4d78-83b0-6b719d94ff05 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:03.493450 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7144c8b5-3a08-4ebc-a510-b21b331ce852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:03.494086 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7144c8b5-3a08-4ebc-a510-b21b331ce852 admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:03.494367 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7144c8b5-3a08-4ebc-a510-b21b331ce852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:03.494642 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7144c8b5-3a08-4ebc-a510-b21b331ce852 admin admin] PATH_INFO: `/v3/projects/fde2258ae7da4ce6804d78e1e56701f3/users/e11b78ec12274a71b8968991aaa7004a/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:03.509694 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7144c8b5-3a08-4ebc-a510-b21b331ce852 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fde2258ae7da4ce6804d78e1e56701f3, user_id=e11b78ec12274a71b8968991aaa7004a, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:03.512378 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7144c8b5-3a08-4ebc-a510-b21b331ce852 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:03.525125 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1331/2672] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:22:03 2022] PUT /identity/v3/projects/fde2258ae7da4ce6804d78e1e56701f3/users/e11b78ec12274a71b8968991aaa7004a/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:03.535270 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8788fe-67c1-473a-a30f-77e779bc8868 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:03.535864 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8788fe-67c1-473a-a30f-77e779bc8868 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:03.535864 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8788fe-67c1-473a-a30f-77e779bc8868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:03.567478 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-2a8788fe-67c1-473a-a30f-77e779bc8868 None None] MFA Rules not processed for user `e11b78ec12274a71b8968991aaa7004a`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:03.568733 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-2a8788fe-67c1-473a-a30f-77e779bc8868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:03.638894 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1342/2673] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:22:03.651941 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7144c8b5-3a08-4ebc-a510-b21b331ce852 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:03.653772 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-7144c8b5-3a08-4ebc-a510-b21b331ce852 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:03.728719 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7144c8b5-3a08-4ebc-a510-b21b331ce852 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:03.734431 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dae856-67a2-4e40-9cd4-ec6c1744289c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae643cfdcb164aa9b474747cd21aa116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:03.735043 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dae856-67a2-4e40-9cd4-ec6c1744289c service nova] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:03.735325 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dae856-67a2-4e40-9cd4-ec6c1744289c service nova] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:03.736254 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dae856-67a2-4e40-9cd4-ec6c1744289c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:03.744773 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dae856-67a2-4e40-9cd4-ec6c1744289c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:03.746376 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dae856-67a2-4e40-9cd4-ec6c1744289c service nova] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:03.755437 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1332/2674] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:22:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:22:03.904079 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8788fe-67c1-473a-a30f-77e779bc8868 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:03.906219 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-2a8788fe-67c1-473a-a30f-77e779bc8868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:03.988980 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dae856-67a2-4e40-9cd4-ec6c1744289c service nova] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:03.996796 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8788fe-67c1-473a-a30f-77e779bc8868 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:04.001501 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dae856-67a2-4e40-9cd4-ec6c1744289c service nova] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:04.001924 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1d5de5-23cf-4a0c-8af9-c745b144898c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:04.002667 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1d5de5-23cf-4a0c-8af9-c745b144898c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:04.002948 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1d5de5-23cf-4a0c-8af9-c745b144898c admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:04.003225 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1d5de5-23cf-4a0c-8af9-c745b144898c admin admin] PATH_INFO: `/v3/users/a90a6c4d42ca4f14b9baf123eb4d6fa2` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:04.004505 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1d5de5-23cf-4a0c-8af9-c745b144898c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a90a6c4d42ca4f14b9baf123eb4d6fa2)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:04.006445 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1d5de5-23cf-4a0c-8af9-c745b144898c admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:04.006574 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb812db-4831-4e4a-b3ba-0ef04641151a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:04.007696 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb812db-4831-4e4a-b3ba-0ef04641151a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:04.007960 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb812db-4831-4e4a-b3ba-0ef04641151a admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:04.008523 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb812db-4831-4e4a-b3ba-0ef04641151a admin admin] PATH_INFO: `/v3/users/e11b78ec12274a71b8968991aaa7004a` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:04.009682 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb812db-4831-4e4a-b3ba-0ef04641151a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e11b78ec12274a71b8968991aaa7004a)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:04.011603 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb812db-4831-4e4a-b3ba-0ef04641151a admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:04.059250 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-4a1d5de5-23cf-4a0c-8af9-c745b144898c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a90a6c4d42ca4f14b9baf123eb4d6fa2'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:04.059449 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-7eb812db-4831-4e4a-b3ba-0ef04641151a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e11b78ec12274a71b8968991aaa7004a'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:04.061115 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:04.074173 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:04.074383 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-4a1d5de5-23cf-4a0c-8af9-c745b144898c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a90a6c4d42ca4f14b9baf123eb4d6fa2'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:04.075526 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-7eb812db-4831-4e4a-b3ba-0ef04641151a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e11b78ec12274a71b8968991aaa7004a'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:04.082918 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-4a1d5de5-23cf-4a0c-8af9-c745b144898c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a90a6c4d42ca4f14b9baf123eb4d6fa2'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:04.085552 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-7eb812db-4831-4e4a-b3ba-0ef04641151a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e11b78ec12274a71b8968991aaa7004a'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:04.086367 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-7eb812db-4831-4e4a-b3ba-0ef04641151a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e11b78ec12274a71b8968991aaa7004a'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:04.091647 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-4a1d5de5-23cf-4a0c-8af9-c745b144898c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a90a6c4d42ca4f14b9baf123eb4d6fa2'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:04.092375 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-4a1d5de5-23cf-4a0c-8af9-c745b144898c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a90a6c4d42ca4f14b9baf123eb4d6fa2'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:04.095175 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-7eb812db-4831-4e4a-b3ba-0ef04641151a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e11b78ec12274a71b8968991aaa7004a'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:04.099323 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1343/2675] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:22:03 2022] DELETE /identity/v3/users/a90a6c4d42ca4f14b9baf123eb4d6fa2 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:04.101757 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1333/2676] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:22:03 2022] DELETE /identity/v3/users/e11b78ec12274a71b8968991aaa7004a => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:04.103909 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a97ad53-6c94-41c1-9df4-5a0daaa7d697 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:04.104216 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a97ad53-6c94-41c1-9df4-5a0daaa7d697 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:04.104570 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a97ad53-6c94-41c1-9df4-5a0daaa7d697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:04.106233 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff80d7f8-0682-446f-a3a5-1839a98b5fc9 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:04.106523 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff80d7f8-0682-446f-a3a5-1839a98b5fc9 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:04.106806 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff80d7f8-0682-446f-a3a5-1839a98b5fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:04.125344 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-3a97ad53-6c94-41c1-9df4-5a0daaa7d697 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:04.126548 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-3a97ad53-6c94-41c1-9df4-5a0daaa7d697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:04.126900 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-ff80d7f8-0682-446f-a3a5-1839a98b5fc9 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:04.128091 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-ff80d7f8-0682-446f-a3a5-1839a98b5fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:04.188059 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1334/2677] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:22:04.196936 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1344/2678] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:22:04.202605 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff80d7f8-0682-446f-a3a5-1839a98b5fc9 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:04.204152 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-ff80d7f8-0682-446f-a3a5-1839a98b5fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:04.207398 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a97ad53-6c94-41c1-9df4-5a0daaa7d697 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:04.208778 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-3a97ad53-6c94-41c1-9df4-5a0daaa7d697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:04.266571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff80d7f8-0682-446f-a3a5-1839a98b5fc9 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:04.270959 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73793aec-11dd-4051-9a30-71edd1c7ce77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:04.271579 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73793aec-11dd-4051-9a30-71edd1c7ce77 service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:04.271865 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73793aec-11dd-4051-9a30-71edd1c7ce77 service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:04.272176 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73793aec-11dd-4051-9a30-71edd1c7ce77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:04.274075 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a97ad53-6c94-41c1-9df4-5a0daaa7d697 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:04.277700 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf03e474-5d7b-4adf-b4f8-746ff2da92bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:04.278335 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf03e474-5d7b-4adf-b4f8-746ff2da92bd service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:04.278610 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf03e474-5d7b-4adf-b4f8-746ff2da92bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:04.278877 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf03e474-5d7b-4adf-b4f8-746ff2da92bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:04.281642 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73793aec-11dd-4051-9a30-71edd1c7ce77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:04.283112 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73793aec-11dd-4051-9a30-71edd1c7ce77 service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:04.290094 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf03e474-5d7b-4adf-b4f8-746ff2da92bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:04.291837 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf03e474-5d7b-4adf-b4f8-746ff2da92bd service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:04.294305 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1335/2679] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:22:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:22:04.305777 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1345/2680] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:22:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 01:22:04.589309 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73793aec-11dd-4051-9a30-71edd1c7ce77 service neutron] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:04.593304 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-73793aec-11dd-4051-9a30-71edd1c7ce77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:04.594235 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf03e474-5d7b-4adf-b4f8-746ff2da92bd service neutron] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:04.596322 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-bf03e474-5d7b-4adf-b4f8-746ff2da92bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:04.612027 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf03e474-5d7b-4adf-b4f8-746ff2da92bd service neutron] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:04.623407 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e9e11d-f39e-4455-9c5a-914b3f725b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:04.623695 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9e11d-f39e-4455-9c5a-914b3f725b53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:04.623980 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9e11d-f39e-4455-9c5a-914b3f725b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:04.624559 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9e11d-f39e-4455-9c5a-914b3f725b53 admin admin] PATH_INFO: `/v3/projects/5f8e546223a549a1a9997c146fbf6d4c` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:04.625793 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e9e11d-f39e-4455-9c5a-914b3f725b53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f8e546223a549a1a9997c146fbf6d4c)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:04.625983 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73793aec-11dd-4051-9a30-71edd1c7ce77 service neutron] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:04.627709 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e9e11d-f39e-4455-9c5a-914b3f725b53 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:04.631671 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe4659b-be40-462f-a39b-985d0a982841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:04.633515 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe4659b-be40-462f-a39b-985d0a982841 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:04.634047 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe4659b-be40-462f-a39b-985d0a982841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:04.634413 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe4659b-be40-462f-a39b-985d0a982841 admin admin] PATH_INFO: `/v3/projects/fde2258ae7da4ce6804d78e1e56701f3` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:04.636174 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe4659b-be40-462f-a39b-985d0a982841 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fde2258ae7da4ce6804d78e1e56701f3)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:04.638466 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe4659b-be40-462f-a39b-985d0a982841 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:04.685838 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-02e9e11d-f39e-4455-9c5a-914b3f725b53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f8e546223a549a1a9997c146fbf6d4c', 'request_id': 'req-02e9e11d-f39e-4455-9c5a-914b3f725b53'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:04.696331 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-02e9e11d-f39e-4455-9c5a-914b3f725b53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f8e546223a549a1a9997c146fbf6d4c', 'request_id': 'req-02e9e11d-f39e-4455-9c5a-914b3f725b53'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:04.703203 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-02e9e11d-f39e-4455-9c5a-914b3f725b53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f8e546223a549a1a9997c146fbf6d4c', 'request_id': 'req-02e9e11d-f39e-4455-9c5a-914b3f725b53'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:04.705149 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:04.721057 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-02e9e11d-f39e-4455-9c5a-914b3f725b53 admin admin] The token cache is being invalidate because project 5f8e546223a549a1a9997c146fbf6d4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:22:04.721351 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-02e9e11d-f39e-4455-9c5a-914b3f725b53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:04.722055 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-02e9e11d-f39e-4455-9c5a-914b3f725b53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:04.723573 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1346/2681] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:22:04 2022] DELETE /identity/v3/projects/5f8e546223a549a1a9997c146fbf6d4c => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:22:04.823519 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-4fe4659b-be40-462f-a39b-985d0a982841 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fde2258ae7da4ce6804d78e1e56701f3', 'request_id': 'req-4fe4659b-be40-462f-a39b-985d0a982841'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:04.832348 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-4fe4659b-be40-462f-a39b-985d0a982841 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fde2258ae7da4ce6804d78e1e56701f3', 'request_id': 'req-4fe4659b-be40-462f-a39b-985d0a982841'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:04.840311 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-4fe4659b-be40-462f-a39b-985d0a982841 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fde2258ae7da4ce6804d78e1e56701f3', 'request_id': 'req-4fe4659b-be40-462f-a39b-985d0a982841'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:04.852525 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-4fe4659b-be40-462f-a39b-985d0a982841 admin admin] The token cache is being invalidate because project fde2258ae7da4ce6804d78e1e56701f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81661) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:22:04.852819 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-4fe4659b-be40-462f-a39b-985d0a982841 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:04.853495 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-4fe4659b-be40-462f-a39b-985d0a982841 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:04.855007 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1336/2682] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:22:04 2022] DELETE /identity/v3/projects/fde2258ae7da4ce6804d78e1e56701f3 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:04.860046 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde88335-8b73-4ba7-9068-dfa56d687998 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:04.860358 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde88335-8b73-4ba7-9068-dfa56d687998 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:04.860659 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde88335-8b73-4ba7-9068-dfa56d687998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:04.882963 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-fde88335-8b73-4ba7-9068-dfa56d687998 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:04.884525 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-fde88335-8b73-4ba7-9068-dfa56d687998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:04.950512 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1347/2683] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:22:04.958766 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965efbba-4dfa-453a-a40b-48958205c3a7 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:04.959143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965efbba-4dfa-453a-a40b-48958205c3a7 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:04.959507 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965efbba-4dfa-453a-a40b-48958205c3a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:04.976852 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:04.983029 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-965efbba-4dfa-453a-a40b-48958205c3a7 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:04.984794 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-965efbba-4dfa-453a-a40b-48958205c3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:04.996598 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1337/2684] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:22:04.997760 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde88335-8b73-4ba7-9068-dfa56d687998 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:05.008663 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde88335-8b73-4ba7-9068-dfa56d687998 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:05.013323 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20006694-a9a3-489e-a0ad-0e333d5c690b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:05.014030 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20006694-a9a3-489e-a0ad-0e333d5c690b admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:05.014310 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20006694-a9a3-489e-a0ad-0e333d5c690b admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:05.014582 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20006694-a9a3-489e-a0ad-0e333d5c690b admin admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:05.015228 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20006694-a9a3-489e-a0ad-0e333d5c690b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:22:05.015532 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20006694-a9a3-489e-a0ad-0e333d5c690b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:05.017141 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20006694-a9a3-489e-a0ad-0e333d5c690b admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:05.029562 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1348/2685] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:22:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:22:05.034177 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965efbba-4dfa-453a-a40b-48958205c3a7 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:05.043355 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965efbba-4dfa-453a-a40b-48958205c3a7 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:05.047951 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e780c4-896f-4fa7-83fd-ee967e1a15b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:05.048579 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e780c4-896f-4fa7-83fd-ee967e1a15b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:05.048855 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e780c4-896f-4fa7-83fd-ee967e1a15b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:05.049120 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e780c4-896f-4fa7-83fd-ee967e1a15b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:05.049747 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e780c4-896f-4fa7-83fd-ee967e1a15b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:05.051661 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e780c4-896f-4fa7-83fd-ee967e1a15b9 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:05.079813 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1338/2686] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:22:05 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:22:05.084471 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20006694-a9a3-489e-a0ad-0e333d5c690b admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:05.095822 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20006694-a9a3-489e-a0ad-0e333d5c690b admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:05.153642 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92284498-3bab-4ecb-b97f-c651fbd3e1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:05.154269 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92284498-3bab-4ecb-b97f-c651fbd3e1d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:05.154546 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92284498-3bab-4ecb-b97f-c651fbd3e1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:05.154812 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92284498-3bab-4ecb-b97f-c651fbd3e1d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:05.155432 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92284498-3bab-4ecb-b97f-c651fbd3e1d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:05.157402 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92284498-3bab-4ecb-b97f-c651fbd3e1d3 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:05.178842 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1349/2687] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:22:05 2022] POST /identity/v3/users => generated 283 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:22:05.183172 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e780c4-896f-4fa7-83fd-ee967e1a15b9 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:05.195989 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e780c4-896f-4fa7-83fd-ee967e1a15b9 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:05.202342 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e93ac-f2b0-4056-b98f-684748085bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:05.202957 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e93ac-f2b0-4056-b98f-684748085bc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:05.203244 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e93ac-f2b0-4056-b98f-684748085bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:05.203503 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e93ac-f2b0-4056-b98f-684748085bc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:05.204184 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567e93ac-f2b0-4056-b98f-684748085bc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:05.205786 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567e93ac-f2b0-4056-b98f-684748085bc5 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:05.217000 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1339/2688] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:22:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:22:05.221131 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92284498-3bab-4ecb-b97f-c651fbd3e1d3 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:05.234702 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92284498-3bab-4ecb-b97f-c651fbd3e1d3 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:05.239493 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07918bdc-bc21-4ffb-b4a2-19a3d79c7b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:05.240595 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07918bdc-bc21-4ffb-b4a2-19a3d79c7b92 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:05.240722 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07918bdc-bc21-4ffb-b4a2-19a3d79c7b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:05.240994 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07918bdc-bc21-4ffb-b4a2-19a3d79c7b92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:05.242257 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07918bdc-bc21-4ffb-b4a2-19a3d79c7b92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:05.243938 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07918bdc-bc21-4ffb-b4a2-19a3d79c7b92 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:05.254970 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1350/2689] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:22:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:22:05.259953 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e93ac-f2b0-4056-b98f-684748085bc5 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:05.273745 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e93ac-f2b0-4056-b98f-684748085bc5 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:05.278598 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd750e08-cbea-4ddd-9db7-44c9324e09ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:05.279265 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd750e08-cbea-4ddd-9db7-44c9324e09ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:05.279567 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd750e08-cbea-4ddd-9db7-44c9324e09ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:05.279834 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd750e08-cbea-4ddd-9db7-44c9324e09ae admin admin] PATH_INFO: `/v3/projects/884fa6321ce142588e8c5521d37fb987/users/fb64acb3ddc44985acba9ea1ff68044e/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:05.294518 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd750e08-cbea-4ddd-9db7-44c9324e09ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=884fa6321ce142588e8c5521d37fb987, user_id=fb64acb3ddc44985acba9ea1ff68044e, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:05.298148 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd750e08-cbea-4ddd-9db7-44c9324e09ae admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:05.311416 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1340/2690] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:22:05 2022] PUT /identity/v3/projects/884fa6321ce142588e8c5521d37fb987/users/fb64acb3ddc44985acba9ea1ff68044e/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:05.323435 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c36db6-48e1-4a49-b660-266fb9c3c47a None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:05.323706 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c36db6-48e1-4a49-b660-266fb9c3c47a None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:05.323976 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c36db6-48e1-4a49-b660-266fb9c3c47a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:05.351244 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-d8c36db6-48e1-4a49-b660-266fb9c3c47a None None] MFA Rules not processed for user `fb64acb3ddc44985acba9ea1ff68044e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:05.352368 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-d8c36db6-48e1-4a49-b660-266fb9c3c47a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:05.398737 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:05.419895 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1351/2691] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:22:05.424502 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd750e08-cbea-4ddd-9db7-44c9324e09ae admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:05.486085 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd750e08-cbea-4ddd-9db7-44c9324e09ae admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:05.491017 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad6145-5f6a-40bf-9347-eec90e0d2720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:05.492188 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad6145-5f6a-40bf-9347-eec90e0d2720 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:05.492463 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad6145-5f6a-40bf-9347-eec90e0d2720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:05.492785 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad6145-5f6a-40bf-9347-eec90e0d2720 admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:05.493410 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fad6145-5f6a-40bf-9347-eec90e0d2720 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:05.495384 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fad6145-5f6a-40bf-9347-eec90e0d2720 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:05.524490 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1341/2692] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:22:05 2022] POST /identity/v3/projects => generated 365 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:22:05.528588 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c36db6-48e1-4a49-b660-266fb9c3c47a None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:05.539980 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c36db6-48e1-4a49-b660-266fb9c3c47a None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:05.599788 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17ca3ac-7a65-41c4-8e6a-2e65dbcaf3fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:05.600421 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17ca3ac-7a65-41c4-8e6a-2e65dbcaf3fa admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:05.600711 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17ca3ac-7a65-41c4-8e6a-2e65dbcaf3fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:05.601004 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17ca3ac-7a65-41c4-8e6a-2e65dbcaf3fa admin admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:05.601718 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17ca3ac-7a65-41c4-8e6a-2e65dbcaf3fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:05.603594 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17ca3ac-7a65-41c4-8e6a-2e65dbcaf3fa admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:05.624982 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1352/2693] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:22:05 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 01:22:05.629018 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad6145-5f6a-40bf-9347-eec90e0d2720 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:05.641040 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad6145-5f6a-40bf-9347-eec90e0d2720 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:05.645788 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5519086-bc69-4eff-86d3-8e226158e889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:05.646391 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5519086-bc69-4eff-86d3-8e226158e889 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:05.646670 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5519086-bc69-4eff-86d3-8e226158e889 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:05.646946 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5519086-bc69-4eff-86d3-8e226158e889 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:05.647634 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5519086-bc69-4eff-86d3-8e226158e889 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:05.649535 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5519086-bc69-4eff-86d3-8e226158e889 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:05.659575 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1342/2694] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:22:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:22:05.663856 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17ca3ac-7a65-41c4-8e6a-2e65dbcaf3fa admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:05.673667 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17ca3ac-7a65-41c4-8e6a-2e65dbcaf3fa admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:05.678342 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddfeca9-d009-4390-afa7-13b2de30024d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:05.679532 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddfeca9-d009-4390-afa7-13b2de30024d admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:05.679874 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddfeca9-d009-4390-afa7-13b2de30024d admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:05.680095 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddfeca9-d009-4390-afa7-13b2de30024d admin admin] PATH_INFO: `/v3/projects/ce852d25957249d18493f2c1b328eff6/users/40da13421b3e4b69bc936c5baa677182/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:05.694570 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddfeca9-d009-4390-afa7-13b2de30024d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce852d25957249d18493f2c1b328eff6, user_id=40da13421b3e4b69bc936c5baa677182, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:05.697301 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddfeca9-d009-4390-afa7-13b2de30024d admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:05.710839 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1353/2695] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:22:05 2022] PUT /identity/v3/projects/ce852d25957249d18493f2c1b328eff6/users/40da13421b3e4b69bc936c5baa677182/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:05.714863 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5519086-bc69-4eff-86d3-8e226158e889 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:05.724725 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5519086-bc69-4eff-86d3-8e226158e889 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:05.780939 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a386207d-abf4-4c42-a90a-d2e2ddf08296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:05.781539 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a386207d-abf4-4c42-a90a-d2e2ddf08296 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:05.781816 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a386207d-abf4-4c42-a90a-d2e2ddf08296 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:05.782084 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a386207d-abf4-4c42-a90a-d2e2ddf08296 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:05.782711 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a386207d-abf4-4c42-a90a-d2e2ddf08296 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:05.784339 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a386207d-abf4-4c42-a90a-d2e2ddf08296 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:05.793854 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1343/2696] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:22:05 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:22:05.798115 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddfeca9-d009-4390-afa7-13b2de30024d admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:05.808535 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddfeca9-d009-4390-afa7-13b2de30024d admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:05.813121 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5834b9-154e-4066-a5c0-ffb882d2dfab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:05.813763 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5834b9-154e-4066-a5c0-ffb882d2dfab admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:05.814042 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5834b9-154e-4066-a5c0-ffb882d2dfab admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:05.814324 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5834b9-154e-4066-a5c0-ffb882d2dfab admin admin] PATH_INFO: `/v3/projects/ce852d25957249d18493f2c1b328eff6/users/40da13421b3e4b69bc936c5baa677182/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:05.816232 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5834b9-154e-4066-a5c0-ffb882d2dfab admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce852d25957249d18493f2c1b328eff6, user_id=40da13421b3e4b69bc936c5baa677182, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:05.818849 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5834b9-154e-4066-a5c0-ffb882d2dfab admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:05.829415 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1354/2697] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:22:05 2022] PUT /identity/v3/projects/ce852d25957249d18493f2c1b328eff6/users/40da13421b3e4b69bc936c5baa677182/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:05.840099 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64121a90-cdfc-4d30-b34c-e2e461e15fb5 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:05.840416 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64121a90-cdfc-4d30-b34c-e2e461e15fb5 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:05.840691 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64121a90-cdfc-4d30-b34c-e2e461e15fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:05.864896 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:05.868029 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-64121a90-cdfc-4d30-b34c-e2e461e15fb5 None None] MFA Rules not processed for user `40da13421b3e4b69bc936c5baa677182`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:05.869955 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-64121a90-cdfc-4d30-b34c-e2e461e15fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:05.937044 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1344/2698] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:05 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:22:05.949261 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5834b9-154e-4066-a5c0-ffb882d2dfab admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:05.950988 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-6d5834b9-154e-4066-a5c0-ffb882d2dfab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:06.009840 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5834b9-154e-4066-a5c0-ffb882d2dfab admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:06.013484 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d07560-e57d-48b8-beb5-59c109abff08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:06.014071 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d07560-e57d-48b8-beb5-59c109abff08 service cinder] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:06.014344 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d07560-e57d-48b8-beb5-59c109abff08 service cinder] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:06.014610 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d07560-e57d-48b8-beb5-59c109abff08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:06.023898 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d07560-e57d-48b8-beb5-59c109abff08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:06.025340 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d07560-e57d-48b8-beb5-59c109abff08 service cinder] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:06.034517 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1355/2699] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:22:05 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:22:06.108679 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64121a90-cdfc-4d30-b34c-e2e461e15fb5 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:06.118789 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64121a90-cdfc-4d30-b34c-e2e461e15fb5 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:06.122359 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c7d76b-4150-4cbd-b03c-a4c4752c9d7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:06.122951 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c7d76b-4150-4cbd-b03c-a4c4752c9d7f service cinder] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:06.123229 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c7d76b-4150-4cbd-b03c-a4c4752c9d7f service cinder] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:06.123505 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c7d76b-4150-4cbd-b03c-a4c4752c9d7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:06.181587 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c7d76b-4150-4cbd-b03c-a4c4752c9d7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:06.183061 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c7d76b-4150-4cbd-b03c-a4c4752c9d7f service cinder] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:06.191906 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1345/2700] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:22:06 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:22:06.457083 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d07560-e57d-48b8-beb5-59c109abff08 service cinder] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:06.458484 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-70d07560-e57d-48b8-beb5-59c109abff08 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:06.503869 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:06.524908 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d07560-e57d-48b8-beb5-59c109abff08 service cinder] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:06.529481 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd8366-f514-4302-bff3-6d47d76d3ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:06.530099 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd8366-f514-4302-bff3-6d47d76d3ace admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:06.530376 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd8366-f514-4302-bff3-6d47d76d3ace admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:06.530647 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd8366-f514-4302-bff3-6d47d76d3ace admin admin] PATH_INFO: `/v3/users/5cafdaf69e304c7389fec32e0698531e` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:06.531750 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcd8366-f514-4302-bff3-6d47d76d3ace admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cafdaf69e304c7389fec32e0698531e)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:06.533712 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcd8366-f514-4302-bff3-6d47d76d3ace admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:06.580085 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-9bcd8366-f514-4302-bff3-6d47d76d3ace admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cafdaf69e304c7389fec32e0698531e'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:06.592412 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-9bcd8366-f514-4302-bff3-6d47d76d3ace admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cafdaf69e304c7389fec32e0698531e'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:06.599246 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-9bcd8366-f514-4302-bff3-6d47d76d3ace admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cafdaf69e304c7389fec32e0698531e'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:06.607308 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-9bcd8366-f514-4302-bff3-6d47d76d3ace admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cafdaf69e304c7389fec32e0698531e'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:06.608069 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-9bcd8366-f514-4302-bff3-6d47d76d3ace admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cafdaf69e304c7389fec32e0698531e'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:06.614497 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1356/2701] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:22:06 2022] DELETE /identity/v3/users/5cafdaf69e304c7389fec32e0698531e => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:06.619719 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42561e69-33ad-4de0-b846-d058a04855d0 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:06.620166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42561e69-33ad-4de0-b846-d058a04855d0 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:06.620434 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42561e69-33ad-4de0-b846-d058a04855d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:06.639785 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-42561e69-33ad-4de0-b846-d058a04855d0 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:06.641027 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-42561e69-33ad-4de0-b846-d058a04855d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:06.698412 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1346/2702] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:22:06.707881 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd8366-f514-4302-bff3-6d47d76d3ace admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:06.709322 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-9bcd8366-f514-4302-bff3-6d47d76d3ace admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:06.766342 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd8366-f514-4302-bff3-6d47d76d3ace admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:06.770016 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b83777b-2403-40ed-92aa-134a04e0fc32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:06.770608 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83777b-2403-40ed-92aa-134a04e0fc32 service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:06.770883 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83777b-2403-40ed-92aa-134a04e0fc32 service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:06.771153 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83777b-2403-40ed-92aa-134a04e0fc32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:06.780346 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b83777b-2403-40ed-92aa-134a04e0fc32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:06.781810 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b83777b-2403-40ed-92aa-134a04e0fc32 service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:06.792408 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1357/2703] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:22:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:22:07.057370 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42561e69-33ad-4de0-b846-d058a04855d0 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:07.058805 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-42561e69-33ad-4de0-b846-d058a04855d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:07.072554 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42561e69-33ad-4de0-b846-d058a04855d0 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:07.077177 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4d1480-1e88-4002-89ef-4bb01d9ec3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:07.077810 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d1480-1e88-4002-89ef-4bb01d9ec3f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:07.078418 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d1480-1e88-4002-89ef-4bb01d9ec3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:07.078418 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d1480-1e88-4002-89ef-4bb01d9ec3f1 admin admin] PATH_INFO: `/v3/projects/c29767f090d84948924ae79b27aa7d6b` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:07.079457 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4d1480-1e88-4002-89ef-4bb01d9ec3f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c29767f090d84948924ae79b27aa7d6b)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:07.081444 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4d1480-1e88-4002-89ef-4bb01d9ec3f1 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:07.117276 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:07.129967 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-2f4d1480-1e88-4002-89ef-4bb01d9ec3f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c29767f090d84948924ae79b27aa7d6b', 'request_id': 'req-2f4d1480-1e88-4002-89ef-4bb01d9ec3f1'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:07.137109 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-2f4d1480-1e88-4002-89ef-4bb01d9ec3f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c29767f090d84948924ae79b27aa7d6b', 'request_id': 'req-2f4d1480-1e88-4002-89ef-4bb01d9ec3f1'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:07.143311 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-2f4d1480-1e88-4002-89ef-4bb01d9ec3f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c29767f090d84948924ae79b27aa7d6b', 'request_id': 'req-2f4d1480-1e88-4002-89ef-4bb01d9ec3f1'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:07.154392 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-2f4d1480-1e88-4002-89ef-4bb01d9ec3f1 admin admin] The token cache is being invalidate because project c29767f090d84948924ae79b27aa7d6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81661) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:22:07.154685 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-2f4d1480-1e88-4002-89ef-4bb01d9ec3f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:07.155504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-2f4d1480-1e88-4002-89ef-4bb01d9ec3f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:07.157013 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1347/2704] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:22:07 2022] DELETE /identity/v3/projects/c29767f090d84948924ae79b27aa7d6b => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:22:07.161002 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b83777b-2403-40ed-92aa-134a04e0fc32 service neutron] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:07.171367 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b83777b-2403-40ed-92aa-134a04e0fc32 service neutron] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:07.228080 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b15581-fbac-437f-8ead-949ec47fb1be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:07.228734 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b15581-fbac-437f-8ead-949ec47fb1be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:07.229015 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b15581-fbac-437f-8ead-949ec47fb1be admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:07.229289 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b15581-fbac-437f-8ead-949ec47fb1be admin admin] PATH_INFO: `/v3/users/3fc092790acd485986651b63912399be` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:07.230401 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b15581-fbac-437f-8ead-949ec47fb1be admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fc092790acd485986651b63912399be)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:07.232367 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b15581-fbac-437f-8ead-949ec47fb1be admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:07.237122 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:07.280706 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-d7b15581-fbac-437f-8ead-949ec47fb1be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fc092790acd485986651b63912399be'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:07.292889 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-d7b15581-fbac-437f-8ead-949ec47fb1be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fc092790acd485986651b63912399be'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:07.299844 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-d7b15581-fbac-437f-8ead-949ec47fb1be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fc092790acd485986651b63912399be'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:07.307301 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-d7b15581-fbac-437f-8ead-949ec47fb1be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fc092790acd485986651b63912399be'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:07.308079 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-d7b15581-fbac-437f-8ead-949ec47fb1be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fc092790acd485986651b63912399be'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:07.314430 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1358/2705] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:22:07 2022] DELETE /identity/v3/users/3fc092790acd485986651b63912399be => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:07.621542 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4d1480-1e88-4002-89ef-4bb01d9ec3f1 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:07.623044 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-2f4d1480-1e88-4002-89ef-4bb01d9ec3f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:07.652606 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4d1480-1e88-4002-89ef-4bb01d9ec3f1 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:07.709730 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53af538-9046-4fdb-b245-9efc74272b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:07.711390 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53af538-9046-4fdb-b245-9efc74272b06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:07.711654 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53af538-9046-4fdb-b245-9efc74272b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:07.711914 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53af538-9046-4fdb-b245-9efc74272b06 admin admin] PATH_INFO: `/v3/projects/f81400a930704b158c9cc09545e58aea` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:07.713170 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53af538-9046-4fdb-b245-9efc74272b06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f81400a930704b158c9cc09545e58aea)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:07.715211 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53af538-9046-4fdb-b245-9efc74272b06 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:07.766222 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-e53af538-9046-4fdb-b245-9efc74272b06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f81400a930704b158c9cc09545e58aea', 'request_id': 'req-e53af538-9046-4fdb-b245-9efc74272b06'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:07.773794 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-e53af538-9046-4fdb-b245-9efc74272b06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f81400a930704b158c9cc09545e58aea', 'request_id': 'req-e53af538-9046-4fdb-b245-9efc74272b06'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:07.780487 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-e53af538-9046-4fdb-b245-9efc74272b06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f81400a930704b158c9cc09545e58aea', 'request_id': 'req-e53af538-9046-4fdb-b245-9efc74272b06'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:07.791935 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-e53af538-9046-4fdb-b245-9efc74272b06 admin admin] The token cache is being invalidate because project f81400a930704b158c9cc09545e58aea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81661) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:22:07.792252 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-e53af538-9046-4fdb-b245-9efc74272b06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:07.792922 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-e53af538-9046-4fdb-b245-9efc74272b06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:07.794398 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1348/2706] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:22:07 2022] DELETE /identity/v3/projects/f81400a930704b158c9cc09545e58aea => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:07.800358 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef63e3be-2b79-48cd-b0c1-b94c3935f81a None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:07.800894 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef63e3be-2b79-48cd-b0c1-b94c3935f81a None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:07.801197 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef63e3be-2b79-48cd-b0c1-b94c3935f81a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:07.823153 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-ef63e3be-2b79-48cd-b0c1-b94c3935f81a None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:07.824575 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-ef63e3be-2b79-48cd-b0c1-b94c3935f81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:07.890288 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1359/2707] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:22:07.898388 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ccd30-379a-4c21-ae7f-005c665734d1 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:07.898600 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ccd30-379a-4c21-ae7f-005c665734d1 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:07.898887 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ccd30-379a-4c21-ae7f-005c665734d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:07.918777 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-be3ccd30-379a-4c21-ae7f-005c665734d1 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:07.919983 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-be3ccd30-379a-4c21-ae7f-005c665734d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:07.927919 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1349/2708] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:22:07.932840 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef63e3be-2b79-48cd-b0c1-b94c3935f81a None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:07.942561 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef63e3be-2b79-48cd-b0c1-b94c3935f81a None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:07.947341 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210ad035-a9cb-49bc-a333-bb5a3311af1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:07.947946 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210ad035-a9cb-49bc-a333-bb5a3311af1e admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:07.948257 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210ad035-a9cb-49bc-a333-bb5a3311af1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:07.948531 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210ad035-a9cb-49bc-a333-bb5a3311af1e admin admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:07.949175 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210ad035-a9cb-49bc-a333-bb5a3311af1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:22:07.949474 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210ad035-a9cb-49bc-a333-bb5a3311af1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:07.951060 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210ad035-a9cb-49bc-a333-bb5a3311af1e admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:07.963331 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1360/2709] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:22:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:22:07.967634 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3ccd30-379a-4c21-ae7f-005c665734d1 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:07.977332 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3ccd30-379a-4c21-ae7f-005c665734d1 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:07.981982 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8b86b0-afa3-43a9-bd77-b3e484d832e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:07.982568 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b86b0-afa3-43a9-bd77-b3e484d832e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:07.982997 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b86b0-afa3-43a9-bd77-b3e484d832e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:07.983139 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b86b0-afa3-43a9-bd77-b3e484d832e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:07.983735 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8b86b0-afa3-43a9-bd77-b3e484d832e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:07.985677 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8b86b0-afa3-43a9-bd77-b3e484d832e1 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:08.015315 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1350/2710] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:22:07 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:22:08.019821 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210ad035-a9cb-49bc-a333-bb5a3311af1e admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:08.030456 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210ad035-a9cb-49bc-a333-bb5a3311af1e admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:08.086805 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bfe784-14f5-4ea9-9e05-a0fb19e3b0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:08.087403 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bfe784-14f5-4ea9-9e05-a0fb19e3b0cb admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:08.087681 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bfe784-14f5-4ea9-9e05-a0fb19e3b0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:08.087948 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bfe784-14f5-4ea9-9e05-a0fb19e3b0cb admin admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:08.088620 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bfe784-14f5-4ea9-9e05-a0fb19e3b0cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:08.090519 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bfe784-14f5-4ea9-9e05-a0fb19e3b0cb admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:08.106938 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:08.112588 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1361/2711] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:22:08 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:22:08.117464 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8b86b0-afa3-43a9-bd77-b3e484d832e1 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:08.127678 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8b86b0-afa3-43a9-bd77-b3e484d832e1 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:08.133114 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8140c5-ce46-4397-9639-325c825e3095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:08.133985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8140c5-ce46-4397-9639-325c825e3095 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:08.134256 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8140c5-ce46-4397-9639-325c825e3095 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:08.134525 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8140c5-ce46-4397-9639-325c825e3095 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:08.135200 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8140c5-ce46-4397-9639-325c825e3095 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:08.136881 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8140c5-ce46-4397-9639-325c825e3095 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:08.146684 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1351/2712] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:22:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:22:08.151202 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bfe784-14f5-4ea9-9e05-a0fb19e3b0cb admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:08.160951 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bfe784-14f5-4ea9-9e05-a0fb19e3b0cb admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:08.165879 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fb3046-b69c-40cb-a06d-ed2012c32b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:08.166496 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb3046-b69c-40cb-a06d-ed2012c32b3f admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:08.166775 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb3046-b69c-40cb-a06d-ed2012c32b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:08.167040 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb3046-b69c-40cb-a06d-ed2012c32b3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:08.167711 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fb3046-b69c-40cb-a06d-ed2012c32b3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:08.169367 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fb3046-b69c-40cb-a06d-ed2012c32b3f admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:08.180106 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1362/2713] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:22:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 01:22:08.184243 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8140c5-ce46-4397-9639-325c825e3095 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:08.195495 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8140c5-ce46-4397-9639-325c825e3095 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:08.199823 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609eacd4-ddd2-4a43-988e-8decaa844dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:08.200502 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609eacd4-ddd2-4a43-988e-8decaa844dea admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:08.200782 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609eacd4-ddd2-4a43-988e-8decaa844dea admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:08.201055 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609eacd4-ddd2-4a43-988e-8decaa844dea admin admin] PATH_INFO: `/v3/projects/4a5c20d6425142189a83939254e5e3d1/users/9f416dd9590b49539e24c1f2642df055/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:08.209238 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:08.217524 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609eacd4-ddd2-4a43-988e-8decaa844dea admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a5c20d6425142189a83939254e5e3d1, user_id=9f416dd9590b49539e24c1f2642df055, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:08.220983 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609eacd4-ddd2-4a43-988e-8decaa844dea admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:08.235044 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1352/2714] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:22:08 2022] PUT /identity/v3/projects/4a5c20d6425142189a83939254e5e3d1/users/9f416dd9590b49539e24c1f2642df055/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:08.245415 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49256507-63c4-4853-b674-c526b7bc5c31 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:08.245711 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49256507-63c4-4853-b674-c526b7bc5c31 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:08.245971 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49256507-63c4-4853-b674-c526b7bc5c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:08.274927 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-49256507-63c4-4853-b674-c526b7bc5c31 None None] MFA Rules not processed for user `9f416dd9590b49539e24c1f2642df055`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:08.276159 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-49256507-63c4-4853-b674-c526b7bc5c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:08.345182 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1363/2715] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:22:08.349658 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609eacd4-ddd2-4a43-988e-8decaa844dea admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:08.412108 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609eacd4-ddd2-4a43-988e-8decaa844dea admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:08.416914 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d7018-0c21-4312-a9c1-fea583a013a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:08.417506 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d7018-0c21-4312-a9c1-fea583a013a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:08.417780 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d7018-0c21-4312-a9c1-fea583a013a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:08.418046 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d7018-0c21-4312-a9c1-fea583a013a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:08.418669 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07d7018-0c21-4312-a9c1-fea583a013a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:08.420608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07d7018-0c21-4312-a9c1-fea583a013a9 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:08.447203 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1353/2716] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:22:08 2022] POST /identity/v3/projects => generated 387 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:22:08.451288 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49256507-63c4-4853-b674-c526b7bc5c31 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:08.462020 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49256507-63c4-4853-b674-c526b7bc5c31 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:08.520717 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd59a550-1ae0-42db-9e9a-3f2f83e0d759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:08.521312 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59a550-1ae0-42db-9e9a-3f2f83e0d759 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:08.521584 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59a550-1ae0-42db-9e9a-3f2f83e0d759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:08.521848 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59a550-1ae0-42db-9e9a-3f2f83e0d759 admin admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:08.522479 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd59a550-1ae0-42db-9e9a-3f2f83e0d759 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:08.524703 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd59a550-1ae0-42db-9e9a-3f2f83e0d759 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:08.545108 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1364/2717] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:22:08 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:22:08.549256 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d7018-0c21-4312-a9c1-fea583a013a9 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:08.560185 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d7018-0c21-4312-a9c1-fea583a013a9 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:08.564829 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00bc351-d12c-4b21-8eba-f7a3cd7d9a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:08.565420 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00bc351-d12c-4b21-8eba-f7a3cd7d9a32 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:08.565700 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00bc351-d12c-4b21-8eba-f7a3cd7d9a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:08.565975 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00bc351-d12c-4b21-8eba-f7a3cd7d9a32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:08.566615 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00bc351-d12c-4b21-8eba-f7a3cd7d9a32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:08.568258 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00bc351-d12c-4b21-8eba-f7a3cd7d9a32 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:08.579361 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1354/2718] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:22:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 01:22:08.583239 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd59a550-1ae0-42db-9e9a-3f2f83e0d759 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:08.593039 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd59a550-1ae0-42db-9e9a-3f2f83e0d759 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:08.597665 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa732b1e-3462-418f-8655-4aeadadbf154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:08.598243 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa732b1e-3462-418f-8655-4aeadadbf154 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:08.598517 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa732b1e-3462-418f-8655-4aeadadbf154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:08.598784 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa732b1e-3462-418f-8655-4aeadadbf154 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:08.599437 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa732b1e-3462-418f-8655-4aeadadbf154 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:08.601039 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa732b1e-3462-418f-8655-4aeadadbf154 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:08.610689 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1365/2719] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:22:08 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:22:08.614667 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00bc351-d12c-4b21-8eba-f7a3cd7d9a32 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:08.625352 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00bc351-d12c-4b21-8eba-f7a3cd7d9a32 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:08.629987 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab68d6d-1b59-4bb2-9091-4212b112f87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:08.630636 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab68d6d-1b59-4bb2-9091-4212b112f87b admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:08.630916 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab68d6d-1b59-4bb2-9091-4212b112f87b admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:08.631188 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab68d6d-1b59-4bb2-9091-4212b112f87b admin admin] PATH_INFO: `/v3/projects/0de0b244fa484769b3c823a776cd1ec3/users/e008a97d5c964934bd367835f3e4cb6a/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:08.646667 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab68d6d-1b59-4bb2-9091-4212b112f87b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0de0b244fa484769b3c823a776cd1ec3, user_id=e008a97d5c964934bd367835f3e4cb6a, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:08.649482 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab68d6d-1b59-4bb2-9091-4212b112f87b admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:08.662374 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1355/2720] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:22:08 2022] PUT /identity/v3/projects/0de0b244fa484769b3c823a776cd1ec3/users/e008a97d5c964934bd367835f3e4cb6a/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:08.673789 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5246f92c-b159-48bc-8a15-1477e150ef68 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:08.674073 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5246f92c-b159-48bc-8a15-1477e150ef68 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:08.674377 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5246f92c-b159-48bc-8a15-1477e150ef68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:08.703103 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-5246f92c-b159-48bc-8a15-1477e150ef68 None None] MFA Rules not processed for user `e008a97d5c964934bd367835f3e4cb6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:08.704342 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-5246f92c-b159-48bc-8a15-1477e150ef68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:08.749262 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:08.770358 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1366/2721] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:08 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:22:08.775912 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab68d6d-1b59-4bb2-9091-4212b112f87b admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:08.845305 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab68d6d-1b59-4bb2-9091-4212b112f87b admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:08.850027 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816f3e3a-33fc-4361-824e-550dd8adaf59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:08.850654 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816f3e3a-33fc-4361-824e-550dd8adaf59 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:08.850968 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816f3e3a-33fc-4361-824e-550dd8adaf59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:08.851434 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816f3e3a-33fc-4361-824e-550dd8adaf59 admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:08.852154 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816f3e3a-33fc-4361-824e-550dd8adaf59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:08.854075 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816f3e3a-33fc-4361-824e-550dd8adaf59 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:08.881595 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1356/2722] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:22:08 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:22:08.886956 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5246f92c-b159-48bc-8a15-1477e150ef68 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:08.898750 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5246f92c-b159-48bc-8a15-1477e150ef68 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:08.957858 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816f3e3a-33fc-4361-824e-550dd8adaf59 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:08.959713 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-816f3e3a-33fc-4361-824e-550dd8adaf59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:08.961474 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dedf94-04e1-418a-8c71-5bb6d85c6801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:08.962086 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dedf94-04e1-418a-8c71-5bb6d85c6801 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:08.962364 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dedf94-04e1-418a-8c71-5bb6d85c6801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:08.962636 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dedf94-04e1-418a-8c71-5bb6d85c6801 admin admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:08.963276 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dedf94-04e1-418a-8c71-5bb6d85c6801 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:08.965278 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dedf94-04e1-418a-8c71-5bb6d85c6801 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:08.972959 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816f3e3a-33fc-4361-824e-550dd8adaf59 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:08.977832 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efca333-c89f-44ef-a484-a32b4ccda302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:08.978605 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efca333-c89f-44ef-a484-a32b4ccda302 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:08.978933 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efca333-c89f-44ef-a484-a32b4ccda302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:08.979240 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efca333-c89f-44ef-a484-a32b4ccda302 admin admin] PATH_INFO: `/v3/users/fb64acb3ddc44985acba9ea1ff68044e` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:08.980631 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efca333-c89f-44ef-a484-a32b4ccda302 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb64acb3ddc44985acba9ea1ff68044e)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:08.982710 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efca333-c89f-44ef-a484-a32b4ccda302 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:08.990702 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1367/2723] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:22:08 2022] POST /identity/v3/users => generated 299 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:22:08.991742 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dedf94-04e1-418a-8c71-5bb6d85c6801 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:08.999191 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:09.001738 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dedf94-04e1-418a-8c71-5bb6d85c6801 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:09.006505 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab42686-dc1c-4176-b21a-a4366e202008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:09.007954 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab42686-dc1c-4176-b21a-a4366e202008 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:09.008733 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab42686-dc1c-4176-b21a-a4366e202008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:09.008950 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab42686-dc1c-4176-b21a-a4366e202008 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:09.009767 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab42686-dc1c-4176-b21a-a4366e202008 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:09.011816 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab42686-dc1c-4176-b21a-a4366e202008 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:09.023921 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1368/2724] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:22:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 01:22:09.028078 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab42686-dc1c-4176-b21a-a4366e202008 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:09.032952 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-9efca333-c89f-44ef-a484-a32b4ccda302 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb64acb3ddc44985acba9ea1ff68044e'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:09.037707 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab42686-dc1c-4176-b21a-a4366e202008 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:09.042360 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f1930a-b50a-4b4c-85cf-609e4d4b9b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:09.043011 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1930a-b50a-4b4c-85cf-609e4d4b9b01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:09.043292 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1930a-b50a-4b4c-85cf-609e4d4b9b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:09.043565 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1930a-b50a-4b4c-85cf-609e4d4b9b01 admin admin] PATH_INFO: `/v3/projects/17c2be1545a841b6a148c63f98ac7234/users/4f8c0d0925bc45aeb909f1fbf6320174/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:09.045213 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-9efca333-c89f-44ef-a484-a32b4ccda302 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb64acb3ddc44985acba9ea1ff68044e'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:09.052404 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-9efca333-c89f-44ef-a484-a32b4ccda302 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb64acb3ddc44985acba9ea1ff68044e'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:09.053103 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-9efca333-c89f-44ef-a484-a32b4ccda302 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb64acb3ddc44985acba9ea1ff68044e'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:09.058299 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f1930a-b50a-4b4c-85cf-609e4d4b9b01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17c2be1545a841b6a148c63f98ac7234, user_id=4f8c0d0925bc45aeb909f1fbf6320174, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:09.061165 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f1930a-b50a-4b4c-85cf-609e4d4b9b01 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:09.061305 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-9efca333-c89f-44ef-a484-a32b4ccda302 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb64acb3ddc44985acba9ea1ff68044e'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:09.067967 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1357/2725] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:22:08 2022] DELETE /identity/v3/users/fb64acb3ddc44985acba9ea1ff68044e => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:22:09.073071 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32809fd6-f595-4945-84ad-8644301dfc0e None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:09.073350 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32809fd6-f595-4945-84ad-8644301dfc0e None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:09.073641 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32809fd6-f595-4945-84ad-8644301dfc0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:09.075468 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1369/2726] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:22:09 2022] PUT /identity/v3/projects/17c2be1545a841b6a148c63f98ac7234/users/4f8c0d0925bc45aeb909f1fbf6320174/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:22:09.079211 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f1930a-b50a-4b4c-85cf-609e4d4b9b01 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:09.080862 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-f8f1930a-b50a-4b4c-85cf-609e4d4b9b01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:09.093380 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-32809fd6-f595-4945-84ad-8644301dfc0e None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:09.094578 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-32809fd6-f595-4945-84ad-8644301dfc0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:09.145672 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f1930a-b50a-4b4c-85cf-609e4d4b9b01 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:09.150923 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef66e15c-6730-4c8b-979f-ca2b23e6dfea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:09.151525 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef66e15c-6730-4c8b-979f-ca2b23e6dfea admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:09.151814 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef66e15c-6730-4c8b-979f-ca2b23e6dfea admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:09.152084 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef66e15c-6730-4c8b-979f-ca2b23e6dfea admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:09.152750 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef66e15c-6730-4c8b-979f-ca2b23e6dfea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:09.153800 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1358/2727] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:22:09.155096 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef66e15c-6730-4c8b-979f-ca2b23e6dfea admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:09.163414 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32809fd6-f595-4945-84ad-8644301dfc0e None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:09.165154 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-32809fd6-f595-4945-84ad-8644301dfc0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:09.165742 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1370/2728] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 01:22:09.169711 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef66e15c-6730-4c8b-979f-ca2b23e6dfea admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:09.180976 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef66e15c-6730-4c8b-979f-ca2b23e6dfea admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:09.186503 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a45a63-cb46-41a9-8bf3-e504722e58b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:09.187483 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a45a63-cb46-41a9-8bf3-e504722e58b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:09.187706 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a45a63-cb46-41a9-8bf3-e504722e58b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:09.187977 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a45a63-cb46-41a9-8bf3-e504722e58b3 admin admin] PATH_INFO: `/v3/projects/17c2be1545a841b6a148c63f98ac7234/users/4f8c0d0925bc45aeb909f1fbf6320174/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:09.189917 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a45a63-cb46-41a9-8bf3-e504722e58b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17c2be1545a841b6a148c63f98ac7234, user_id=4f8c0d0925bc45aeb909f1fbf6320174, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:09.193414 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a45a63-cb46-41a9-8bf3-e504722e58b3 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:09.204674 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1371/2729] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:22:09 2022] PUT /identity/v3/projects/17c2be1545a841b6a148c63f98ac7234/users/4f8c0d0925bc45aeb909f1fbf6320174/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:09.215133 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af137129-9e88-4b8a-9113-834f25568bbb None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:09.215534 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af137129-9e88-4b8a-9113-834f25568bbb None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:09.215813 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af137129-9e88-4b8a-9113-834f25568bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:09.225177 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32809fd6-f595-4945-84ad-8644301dfc0e None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:09.244697 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-af137129-9e88-4b8a-9113-834f25568bbb None None] MFA Rules not processed for user `4f8c0d0925bc45aeb909f1fbf6320174`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:09.245957 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-af137129-9e88-4b8a-9113-834f25568bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:09.259301 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:09.278088 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe52ad8-9463-4323-9c7d-ab566f6bb7ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:09.278783 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe52ad8-9463-4323-9c7d-ab566f6bb7ee service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:09.279130 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe52ad8-9463-4323-9c7d-ab566f6bb7ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:09.279400 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe52ad8-9463-4323-9c7d-ab566f6bb7ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:09.283688 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:09.318555 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1372/2730] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:09 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:22:09.329988 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af137129-9e88-4b8a-9113-834f25568bbb None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:09.331495 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-af137129-9e88-4b8a-9113-834f25568bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:09.344179 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe52ad8-9463-4323-9c7d-ab566f6bb7ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:09.345662 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe52ad8-9463-4323-9c7d-ab566f6bb7ee service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:09.355408 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1359/2731] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:22:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 01:22:09.391621 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af137129-9e88-4b8a-9113-834f25568bbb None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:09.395290 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ed2414-0a6b-40f3-8acd-4788d2c283c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:09.395894 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ed2414-0a6b-40f3-8acd-4788d2c283c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:09.396193 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ed2414-0a6b-40f3-8acd-4788d2c283c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:09.396470 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ed2414-0a6b-40f3-8acd-4788d2c283c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:09.405689 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ed2414-0a6b-40f3-8acd-4788d2c283c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:09.407118 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ed2414-0a6b-40f3-8acd-4788d2c283c7 service cinder] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:09.418278 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1373/2732] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:22:09 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 01:22:09.633586 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe52ad8-9463-4323-9c7d-ab566f6bb7ee service neutron] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:09.635409 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-bbe52ad8-9463-4323-9c7d-ab566f6bb7ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:09.655768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe52ad8-9463-4323-9c7d-ab566f6bb7ee service neutron] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:09.660998 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f99acc-27cf-49fb-90f6-5bffbcfaaab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:09.661782 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f99acc-27cf-49fb-90f6-5bffbcfaaab4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:09.662184 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f99acc-27cf-49fb-90f6-5bffbcfaaab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:09.662556 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f99acc-27cf-49fb-90f6-5bffbcfaaab4 admin admin] PATH_INFO: `/v3/projects/884fa6321ce142588e8c5521d37fb987` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:09.664695 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f99acc-27cf-49fb-90f6-5bffbcfaaab4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=884fa6321ce142588e8c5521d37fb987)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:09.666915 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f99acc-27cf-49fb-90f6-5bffbcfaaab4 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:09.716695 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-a5f99acc-27cf-49fb-90f6-5bffbcfaaab4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '884fa6321ce142588e8c5521d37fb987', 'request_id': 'req-a5f99acc-27cf-49fb-90f6-5bffbcfaaab4'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:09.724810 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-a5f99acc-27cf-49fb-90f6-5bffbcfaaab4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '884fa6321ce142588e8c5521d37fb987', 'request_id': 'req-a5f99acc-27cf-49fb-90f6-5bffbcfaaab4'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:09.732227 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-a5f99acc-27cf-49fb-90f6-5bffbcfaaab4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '884fa6321ce142588e8c5521d37fb987', 'request_id': 'req-a5f99acc-27cf-49fb-90f6-5bffbcfaaab4'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:09.743263 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-a5f99acc-27cf-49fb-90f6-5bffbcfaaab4 admin admin] The token cache is being invalidate because project 884fa6321ce142588e8c5521d37fb987 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81661) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:22:09.743558 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-a5f99acc-27cf-49fb-90f6-5bffbcfaaab4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:09.744319 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-a5f99acc-27cf-49fb-90f6-5bffbcfaaab4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:09.745857 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1360/2733] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:22:09 2022] DELETE /identity/v3/projects/884fa6321ce142588e8c5521d37fb987 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:09.750094 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ed2414-0a6b-40f3-8acd-4788d2c283c7 service cinder] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:09.762211 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ed2414-0a6b-40f3-8acd-4788d2c283c7 service cinder] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:09.825083 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d256fa48-3b7d-449a-9267-f552789c4c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:09.825717 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256fa48-3b7d-449a-9267-f552789c4c46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:09.825999 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256fa48-3b7d-449a-9267-f552789c4c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:09.826277 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256fa48-3b7d-449a-9267-f552789c4c46 admin admin] PATH_INFO: `/v3/users/40da13421b3e4b69bc936c5baa677182` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:09.827401 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d256fa48-3b7d-449a-9267-f552789c4c46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40da13421b3e4b69bc936c5baa677182)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:09.829398 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d256fa48-3b7d-449a-9267-f552789c4c46 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:09.879014 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-d256fa48-3b7d-449a-9267-f552789c4c46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40da13421b3e4b69bc936c5baa677182'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:09.885031 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:09.895086 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-d256fa48-3b7d-449a-9267-f552789c4c46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40da13421b3e4b69bc936c5baa677182'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:09.903047 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-d256fa48-3b7d-449a-9267-f552789c4c46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40da13421b3e4b69bc936c5baa677182'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:09.911074 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-d256fa48-3b7d-449a-9267-f552789c4c46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40da13421b3e4b69bc936c5baa677182'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:09.912047 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-d256fa48-3b7d-449a-9267-f552789c4c46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40da13421b3e4b69bc936c5baa677182'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:09.918569 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1374/2734] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:22:09 2022] DELETE /identity/v3/users/40da13421b3e4b69bc936c5baa677182 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:09.953066 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f99acc-27cf-49fb-90f6-5bffbcfaaab4 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:09.954516 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-a5f99acc-27cf-49fb-90f6-5bffbcfaaab4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:10.018517 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f99acc-27cf-49fb-90f6-5bffbcfaaab4 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:10.057268 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:10.071272 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06db8b2-166e-400b-9f8a-f2cee36d860f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:10.072600 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06db8b2-166e-400b-9f8a-f2cee36d860f service cinder] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:10.073154 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06db8b2-166e-400b-9f8a-f2cee36d860f service cinder] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:10.073874 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06db8b2-166e-400b-9f8a-f2cee36d860f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:10.076166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-d06db8b2-166e-400b-9f8a-f2cee36d860f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:10.138661 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06db8b2-166e-400b-9f8a-f2cee36d860f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:10.140202 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06db8b2-166e-400b-9f8a-f2cee36d860f service cinder] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:10.149484 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1361/2735] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:22:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:22:10.196659 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d256fa48-3b7d-449a-9267-f552789c4c46 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:10.198111 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-d256fa48-3b7d-449a-9267-f552789c4c46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:10.268919 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d256fa48-3b7d-449a-9267-f552789c4c46 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:10.274264 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a41387-c281-43f6-a4f0-305cfb269e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:10.274898 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a41387-c281-43f6-a4f0-305cfb269e58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:10.275175 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a41387-c281-43f6-a4f0-305cfb269e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:10.275447 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a41387-c281-43f6-a4f0-305cfb269e58 admin admin] PATH_INFO: `/v3/projects/ce852d25957249d18493f2c1b328eff6` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:10.276597 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a41387-c281-43f6-a4f0-305cfb269e58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce852d25957249d18493f2c1b328eff6)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:10.278504 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a41387-c281-43f6-a4f0-305cfb269e58 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:10.332026 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-e1a41387-c281-43f6-a4f0-305cfb269e58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce852d25957249d18493f2c1b328eff6', 'request_id': 'req-e1a41387-c281-43f6-a4f0-305cfb269e58'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:10.339186 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-e1a41387-c281-43f6-a4f0-305cfb269e58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce852d25957249d18493f2c1b328eff6', 'request_id': 'req-e1a41387-c281-43f6-a4f0-305cfb269e58'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:10.345790 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-e1a41387-c281-43f6-a4f0-305cfb269e58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce852d25957249d18493f2c1b328eff6', 'request_id': 'req-e1a41387-c281-43f6-a4f0-305cfb269e58'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:10.358725 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-e1a41387-c281-43f6-a4f0-305cfb269e58 admin admin] The token cache is being invalidate because project ce852d25957249d18493f2c1b328eff6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:22:10.358940 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-e1a41387-c281-43f6-a4f0-305cfb269e58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:10.359538 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-e1a41387-c281-43f6-a4f0-305cfb269e58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:10.362164 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1375/2736] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:22:10 2022] DELETE /identity/v3/projects/ce852d25957249d18493f2c1b328eff6 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:10.367719 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b359c2ce-1185-4fd3-9cdf-a2926cd6dbd2 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:10.368014 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b359c2ce-1185-4fd3-9cdf-a2926cd6dbd2 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:10.368327 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b359c2ce-1185-4fd3-9cdf-a2926cd6dbd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:10.390291 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-b359c2ce-1185-4fd3-9cdf-a2926cd6dbd2 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:10.391780 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-b359c2ce-1185-4fd3-9cdf-a2926cd6dbd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:10.461430 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1362/2737] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:22:10.478072 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734c631e-f94a-4113-9ef3-5ab4c4a9da47 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:10.478360 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734c631e-f94a-4113-9ef3-5ab4c4a9da47 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:10.478634 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734c631e-f94a-4113-9ef3-5ab4c4a9da47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:10.499352 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-734c631e-f94a-4113-9ef3-5ab4c4a9da47 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:10.500740 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-734c631e-f94a-4113-9ef3-5ab4c4a9da47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:10.509919 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1376/2738] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:22:10.514577 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b359c2ce-1185-4fd3-9cdf-a2926cd6dbd2 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:10.525111 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b359c2ce-1185-4fd3-9cdf-a2926cd6dbd2 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:10.530069 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc4804b-532f-422d-9b98-0e05785cf54f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:10.530673 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc4804b-532f-422d-9b98-0e05785cf54f admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:10.531121 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc4804b-532f-422d-9b98-0e05785cf54f admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:10.531379 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc4804b-532f-422d-9b98-0e05785cf54f admin admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:10.532024 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc4804b-532f-422d-9b98-0e05785cf54f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:22:10.532385 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc4804b-532f-422d-9b98-0e05785cf54f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:10.534032 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc4804b-532f-422d-9b98-0e05785cf54f admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:10.545427 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1363/2739] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:22:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:22:10.549825 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734c631e-f94a-4113-9ef3-5ab4c4a9da47 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:10.558981 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734c631e-f94a-4113-9ef3-5ab4c4a9da47 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:10.563638 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860d974a-2010-4438-8ff1-b5c3878cb608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:10.564358 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d974a-2010-4438-8ff1-b5c3878cb608 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:10.564638 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d974a-2010-4438-8ff1-b5c3878cb608 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:10.564909 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d974a-2010-4438-8ff1-b5c3878cb608 admin admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:10.565528 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860d974a-2010-4438-8ff1-b5c3878cb608 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:10.567415 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860d974a-2010-4438-8ff1-b5c3878cb608 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:10.596692 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1377/2740] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:22:10 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:22:10.600990 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc4804b-532f-422d-9b98-0e05785cf54f admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:10.610788 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc4804b-532f-422d-9b98-0e05785cf54f admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:10.667925 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11550c0-2ebf-4a39-a64c-e8029a6bbbc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:10.668624 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11550c0-2ebf-4a39-a64c-e8029a6bbbc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:10.668823 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11550c0-2ebf-4a39-a64c-e8029a6bbbc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:10.669089 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11550c0-2ebf-4a39-a64c-e8029a6bbbc7 admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:10.669709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11550c0-2ebf-4a39-a64c-e8029a6bbbc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:10.671625 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11550c0-2ebf-4a39-a64c-e8029a6bbbc7 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:10.693185 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1364/2741] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:22:10 2022] POST /identity/v3/users => generated 283 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:22:10.698008 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860d974a-2010-4438-8ff1-b5c3878cb608 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:10.708281 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860d974a-2010-4438-8ff1-b5c3878cb608 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:10.713148 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f012a6fb-5b9f-4d20-b799-e40d1428ebe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:10.713769 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012a6fb-5b9f-4d20-b799-e40d1428ebe2 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:10.714045 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012a6fb-5b9f-4d20-b799-e40d1428ebe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:10.714335 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012a6fb-5b9f-4d20-b799-e40d1428ebe2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:10.714971 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f012a6fb-5b9f-4d20-b799-e40d1428ebe2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:10.716775 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f012a6fb-5b9f-4d20-b799-e40d1428ebe2 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:10.726764 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1378/2742] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:22:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:22:10.730957 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11550c0-2ebf-4a39-a64c-e8029a6bbbc7 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:10.741734 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11550c0-2ebf-4a39-a64c-e8029a6bbbc7 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:10.746508 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c0f58-5c91-4014-af04-761c01266a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:10.747106 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c0f58-5c91-4014-af04-761c01266a93 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:10.747389 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c0f58-5c91-4014-af04-761c01266a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:10.747660 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c0f58-5c91-4014-af04-761c01266a93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:10.748340 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c0f58-5c91-4014-af04-761c01266a93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:10.750208 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c0f58-5c91-4014-af04-761c01266a93 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:10.755434 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:10.762342 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1365/2743] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:22:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:22:10.766641 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f012a6fb-5b9f-4d20-b799-e40d1428ebe2 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:10.776584 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f012a6fb-5b9f-4d20-b799-e40d1428ebe2 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:10.781093 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5e330c-0b5f-4cbf-9ec8-0babe0c47929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:10.781730 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e330c-0b5f-4cbf-9ec8-0babe0c47929 admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:10.782006 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e330c-0b5f-4cbf-9ec8-0babe0c47929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:10.782276 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e330c-0b5f-4cbf-9ec8-0babe0c47929 admin admin] PATH_INFO: `/v3/projects/b080b84e286e4ceebdad70f530f23cce/users/9164fe1724044a38840e2a40c627ca7f/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:10.796509 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5e330c-0b5f-4cbf-9ec8-0babe0c47929 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b080b84e286e4ceebdad70f530f23cce, user_id=9164fe1724044a38840e2a40c627ca7f, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:10.799137 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5e330c-0b5f-4cbf-9ec8-0babe0c47929 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:10.807526 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:10.814086 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1379/2744] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:22:10 2022] PUT /identity/v3/projects/b080b84e286e4ceebdad70f530f23cce/users/9164fe1724044a38840e2a40c627ca7f/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:10.824846 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b05464-a943-4ce5-b68f-4498fb70da1b None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:10.825213 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b05464-a943-4ce5-b68f-4498fb70da1b None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:10.825707 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b05464-a943-4ce5-b68f-4498fb70da1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:10.853361 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-76b05464-a943-4ce5-b68f-4498fb70da1b None None] MFA Rules not processed for user `9164fe1724044a38840e2a40c627ca7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:10.854632 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-76b05464-a943-4ce5-b68f-4498fb70da1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:10.920264 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1366/2745] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:22:10.924932 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5e330c-0b5f-4cbf-9ec8-0babe0c47929 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:10.988153 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5e330c-0b5f-4cbf-9ec8-0babe0c47929 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:10.993484 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba73e04-a5b0-496b-ab32-586f67332225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:10.994082 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba73e04-a5b0-496b-ab32-586f67332225 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:10.994359 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba73e04-a5b0-496b-ab32-586f67332225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:10.994635 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba73e04-a5b0-496b-ab32-586f67332225 admin admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:10.995282 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba73e04-a5b0-496b-ab32-586f67332225 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:10.997214 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba73e04-a5b0-496b-ab32-586f67332225 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:11.024094 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1380/2746] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:22:10 2022] POST /identity/v3/projects => generated 367 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:22:11.028238 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b05464-a943-4ce5-b68f-4498fb70da1b None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:11.039446 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b05464-a943-4ce5-b68f-4498fb70da1b None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:11.093429 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0220a3-9eb1-4139-b88f-0645baea9ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:11.094022 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0220a3-9eb1-4139-b88f-0645baea9ad0 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:11.094307 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0220a3-9eb1-4139-b88f-0645baea9ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:11.094579 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0220a3-9eb1-4139-b88f-0645baea9ad0 admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:11.095201 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0220a3-9eb1-4139-b88f-0645baea9ad0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:11.097159 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0220a3-9eb1-4139-b88f-0645baea9ad0 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:11.118504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1367/2747] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:22:11 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:22:11.122520 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba73e04-a5b0-496b-ab32-586f67332225 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:11.132841 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba73e04-a5b0-496b-ab32-586f67332225 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:11.137642 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a324e2e-1f8d-4fd4-8946-12a3966db37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:11.138229 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a324e2e-1f8d-4fd4-8946-12a3966db37e admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:11.138506 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a324e2e-1f8d-4fd4-8946-12a3966db37e admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:11.138774 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a324e2e-1f8d-4fd4-8946-12a3966db37e admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:11.139406 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a324e2e-1f8d-4fd4-8946-12a3966db37e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:11.141065 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a324e2e-1f8d-4fd4-8946-12a3966db37e admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:11.151698 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1381/2748] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:22:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:22:11.155630 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0220a3-9eb1-4139-b88f-0645baea9ad0 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:11.165987 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0220a3-9eb1-4139-b88f-0645baea9ad0 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:11.170517 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7433fd13-7e0b-426a-8dd7-3b71673691d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:11.171150 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7433fd13-7e0b-426a-8dd7-3b71673691d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:11.171425 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7433fd13-7e0b-426a-8dd7-3b71673691d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:11.171696 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7433fd13-7e0b-426a-8dd7-3b71673691d7 admin admin] PATH_INFO: `/v3/projects/09d2c84ae9ef457a9964b7549c3a2847/users/5ab192f84cca48a7a40415d45e2f949d/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:11.186675 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7433fd13-7e0b-426a-8dd7-3b71673691d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09d2c84ae9ef457a9964b7549c3a2847, user_id=5ab192f84cca48a7a40415d45e2f949d, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:11.189351 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7433fd13-7e0b-426a-8dd7-3b71673691d7 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:11.203006 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1368/2749] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:22:11 2022] PUT /identity/v3/projects/09d2c84ae9ef457a9964b7549c3a2847/users/5ab192f84cca48a7a40415d45e2f949d/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:11.206899 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a324e2e-1f8d-4fd4-8946-12a3966db37e admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:11.216898 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a324e2e-1f8d-4fd4-8946-12a3966db37e admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:11.273515 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40d572d-5e22-4409-bcfa-92a8942dd62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:11.274111 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d572d-5e22-4409-bcfa-92a8942dd62c admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:11.274388 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d572d-5e22-4409-bcfa-92a8942dd62c admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:11.274662 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d572d-5e22-4409-bcfa-92a8942dd62c admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:11.275323 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40d572d-5e22-4409-bcfa-92a8942dd62c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:11.276959 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40d572d-5e22-4409-bcfa-92a8942dd62c admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:11.286623 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1382/2750] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:22:11 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:22:11.291362 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7433fd13-7e0b-426a-8dd7-3b71673691d7 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:11.303459 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7433fd13-7e0b-426a-8dd7-3b71673691d7 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:11.309079 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e006ec-bdf3-4bd4-93e4-6d7b24ac6b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:11.309735 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e006ec-bdf3-4bd4-93e4-6d7b24ac6b00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:11.310017 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e006ec-bdf3-4bd4-93e4-6d7b24ac6b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:11.310305 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e006ec-bdf3-4bd4-93e4-6d7b24ac6b00 admin admin] PATH_INFO: `/v3/projects/09d2c84ae9ef457a9964b7549c3a2847/users/5ab192f84cca48a7a40415d45e2f949d/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:11.312266 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e006ec-bdf3-4bd4-93e4-6d7b24ac6b00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09d2c84ae9ef457a9964b7549c3a2847, user_id=5ab192f84cca48a7a40415d45e2f949d, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:11.314915 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e006ec-bdf3-4bd4-93e4-6d7b24ac6b00 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:11.325467 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1369/2751] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:22:11 2022] PUT /identity/v3/projects/09d2c84ae9ef457a9964b7549c3a2847/users/5ab192f84cca48a7a40415d45e2f949d/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:11.335839 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d91ecf6-15d3-40b6-a067-eb5f8aeda91a None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:11.336249 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d91ecf6-15d3-40b6-a067-eb5f8aeda91a None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:11.336539 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d91ecf6-15d3-40b6-a067-eb5f8aeda91a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:11.364696 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-9d91ecf6-15d3-40b6-a067-eb5f8aeda91a None None] MFA Rules not processed for user `5ab192f84cca48a7a40415d45e2f949d`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:11.365904 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-9d91ecf6-15d3-40b6-a067-eb5f8aeda91a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:11.426068 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:11.447434 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1383/2752] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:11 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:22:11.459653 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e006ec-bdf3-4bd4-93e4-6d7b24ac6b00 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:11.461961 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-24e006ec-bdf3-4bd4-93e4-6d7b24ac6b00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:11.510230 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:11.526644 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e006ec-bdf3-4bd4-93e4-6d7b24ac6b00 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:11.530306 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7ebcfa-c880-4cca-8525-2d76df5818ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:11.530904 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ebcfa-c880-4cca-8525-2d76df5818ec service cinder] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:11.531178 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ebcfa-c880-4cca-8525-2d76df5818ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:11.531475 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ebcfa-c880-4cca-8525-2d76df5818ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:11.541593 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7ebcfa-c880-4cca-8525-2d76df5818ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:11.543058 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7ebcfa-c880-4cca-8525-2d76df5818ec service cinder] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:11.552934 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1370/2753] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:22:11 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:22:11.643287 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d91ecf6-15d3-40b6-a067-eb5f8aeda91a None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:11.644856 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-9d91ecf6-15d3-40b6-a067-eb5f8aeda91a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:11.660744 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d91ecf6-15d3-40b6-a067-eb5f8aeda91a None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:11.664839 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658e010f-f1f7-49a3-b6ba-60ac33a808ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:11.665503 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e010f-f1f7-49a3-b6ba-60ac33a808ff service cinder] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:11.665842 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e010f-f1f7-49a3-b6ba-60ac33a808ff service cinder] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:11.666172 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e010f-f1f7-49a3-b6ba-60ac33a808ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:11.730993 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658e010f-f1f7-49a3-b6ba-60ac33a808ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:11.732483 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658e010f-f1f7-49a3-b6ba-60ac33a808ff service cinder] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:11.741861 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1384/2754] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:22:11 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:22:14.232225 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7ebcfa-c880-4cca-8525-2d76df5818ec service cinder] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:14.240709 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7ebcfa-c880-4cca-8525-2d76df5818ec service cinder] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:14.245081 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784ef401-7895-4b9e-8664-f1871d0aaf87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:14.245664 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ef401-7895-4b9e-8664-f1871d0aaf87 service cinder] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:14.245937 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ef401-7895-4b9e-8664-f1871d0aaf87 service cinder] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:14.246206 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ef401-7895-4b9e-8664-f1871d0aaf87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:14.248060 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-784ef401-7895-4b9e-8664-f1871d0aaf87 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:14.308168 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784ef401-7895-4b9e-8664-f1871d0aaf87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:14.309609 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784ef401-7895-4b9e-8664-f1871d0aaf87 service cinder] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:14.318635 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1371/2755] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:22:14 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:22:16.540324 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658e010f-f1f7-49a3-b6ba-60ac33a808ff service cinder] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:16.540324 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-658e010f-f1f7-49a3-b6ba-60ac33a808ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:16.608651 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658e010f-f1f7-49a3-b6ba-60ac33a808ff service cinder] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:16.613213 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f38fa51-6d76-4d36-b017-896fe02b9836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:16.613842 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38fa51-6d76-4d36-b017-896fe02b9836 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:16.614116 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38fa51-6d76-4d36-b017-896fe02b9836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:16.614388 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38fa51-6d76-4d36-b017-896fe02b9836 admin admin] PATH_INFO: `/v3/users/9f416dd9590b49539e24c1f2642df055` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:16.615519 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f38fa51-6d76-4d36-b017-896fe02b9836 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f416dd9590b49539e24c1f2642df055)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:16.618514 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f38fa51-6d76-4d36-b017-896fe02b9836 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:16.671641 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-2f38fa51-6d76-4d36-b017-896fe02b9836 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f416dd9590b49539e24c1f2642df055'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:16.684339 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784ef401-7895-4b9e-8664-f1871d0aaf87 service cinder] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:16.685084 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-2f38fa51-6d76-4d36-b017-896fe02b9836 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f416dd9590b49539e24c1f2642df055'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:16.692330 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-2f38fa51-6d76-4d36-b017-896fe02b9836 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f416dd9590b49539e24c1f2642df055'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:16.695985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784ef401-7895-4b9e-8664-f1871d0aaf87 service cinder] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:16.700776 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-2f38fa51-6d76-4d36-b017-896fe02b9836 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f416dd9590b49539e24c1f2642df055'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:16.701324 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc65249a-3431-4d4f-8c92-117e9e5103e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:16.701474 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-2f38fa51-6d76-4d36-b017-896fe02b9836 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f416dd9590b49539e24c1f2642df055'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:16.701951 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc65249a-3431-4d4f-8c92-117e9e5103e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:16.702236 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc65249a-3431-4d4f-8c92-117e9e5103e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:16.702510 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc65249a-3431-4d4f-8c92-117e9e5103e9 admin admin] PATH_INFO: `/v3/users/9164fe1724044a38840e2a40c627ca7f` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:16.703628 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc65249a-3431-4d4f-8c92-117e9e5103e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9164fe1724044a38840e2a40c627ca7f)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:16.705608 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc65249a-3431-4d4f-8c92-117e9e5103e9 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:16.708074 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1385/2756] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:22:16 2022] DELETE /identity/v3/users/9f416dd9590b49539e24c1f2642df055 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:16.712664 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61500029-de79-4e65-9bf9-fe2ec6ba9b02 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:16.712941 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61500029-de79-4e65-9bf9-fe2ec6ba9b02 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:16.713242 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61500029-de79-4e65-9bf9-fe2ec6ba9b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:16.734556 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-61500029-de79-4e65-9bf9-fe2ec6ba9b02 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:16.736307 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-61500029-de79-4e65-9bf9-fe2ec6ba9b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:16.742479 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:16.753893 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-dc65249a-3431-4d4f-8c92-117e9e5103e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9164fe1724044a38840e2a40c627ca7f'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:16.767242 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-dc65249a-3431-4d4f-8c92-117e9e5103e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9164fe1724044a38840e2a40c627ca7f'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:16.775051 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-dc65249a-3431-4d4f-8c92-117e9e5103e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9164fe1724044a38840e2a40c627ca7f'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:16.775752 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-dc65249a-3431-4d4f-8c92-117e9e5103e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9164fe1724044a38840e2a40c627ca7f'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:16.783670 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-dc65249a-3431-4d4f-8c92-117e9e5103e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9164fe1724044a38840e2a40c627ca7f'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:16.790096 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1372/2757] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:22:16 2022] DELETE /identity/v3/users/9164fe1724044a38840e2a40c627ca7f => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:22:16.794699 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56954b-032a-4528-bbe7-2725268429fc None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:16.794902 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56954b-032a-4528-bbe7-2725268429fc None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:16.795171 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56954b-032a-4528-bbe7-2725268429fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:16.801932 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1386/2758] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:22:16.812808 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61500029-de79-4e65-9bf9-fe2ec6ba9b02 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:16.814162 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-61500029-de79-4e65-9bf9-fe2ec6ba9b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:16.815327 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-5b56954b-032a-4528-bbe7-2725268429fc None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:16.816524 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-5b56954b-032a-4528-bbe7-2725268429fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:16.865233 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:16.877194 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61500029-de79-4e65-9bf9-fe2ec6ba9b02 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:16.879856 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1373/2759] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:22:16.890621 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b56954b-032a-4528-bbe7-2725268429fc None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:16.891927 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-5b56954b-032a-4528-bbe7-2725268429fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:16.928811 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534e672f-698a-4336-989e-806051ff4771 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:16.930064 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534e672f-698a-4336-989e-806051ff4771 service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:16.930546 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534e672f-698a-4336-989e-806051ff4771 service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:16.930926 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534e672f-698a-4336-989e-806051ff4771 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:16.940556 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534e672f-698a-4336-989e-806051ff4771 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:16.942006 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534e672f-698a-4336-989e-806051ff4771 service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:16.949137 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b56954b-032a-4528-bbe7-2725268429fc None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:16.952516 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1387/2760] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:22:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 01:22:16.952871 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6759dce7-4965-4933-af5b-b562613eb347 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:16.953482 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6759dce7-4965-4933-af5b-b562613eb347 service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:16.953830 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6759dce7-4965-4933-af5b-b562613eb347 service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:16.954109 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6759dce7-4965-4933-af5b-b562613eb347 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:16.963881 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6759dce7-4965-4933-af5b-b562613eb347 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:16.965410 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6759dce7-4965-4933-af5b-b562613eb347 service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:16.975173 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1374/2761] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:22:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 01:22:17.248949 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534e672f-698a-4336-989e-806051ff4771 service neutron] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:17.250632 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-534e672f-698a-4336-989e-806051ff4771 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:17.255274 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6759dce7-4965-4933-af5b-b562613eb347 service neutron] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:17.256982 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-6759dce7-4965-4933-af5b-b562613eb347 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:17.270983 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6759dce7-4965-4933-af5b-b562613eb347 service neutron] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:17.276803 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c87ae40-6496-48e2-a0c5-abe560b25882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:17.277583 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87ae40-6496-48e2-a0c5-abe560b25882 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:17.277929 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87ae40-6496-48e2-a0c5-abe560b25882 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:17.278127 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87ae40-6496-48e2-a0c5-abe560b25882 admin admin] PATH_INFO: `/v3/projects/b080b84e286e4ceebdad70f530f23cce` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:17.279289 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c87ae40-6496-48e2-a0c5-abe560b25882 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b080b84e286e4ceebdad70f530f23cce)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:17.281241 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c87ae40-6496-48e2-a0c5-abe560b25882 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:17.282687 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534e672f-698a-4336-989e-806051ff4771 service neutron] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:17.301861 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f8e86f-82bf-4939-91fb-0a19ee0ea2bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:17.302726 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8e86f-82bf-4939-91fb-0a19ee0ea2bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:17.303051 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8e86f-82bf-4939-91fb-0a19ee0ea2bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:17.303328 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8e86f-82bf-4939-91fb-0a19ee0ea2bb admin admin] PATH_INFO: `/v3/projects/4a5c20d6425142189a83939254e5e3d1` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:17.305218 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f8e86f-82bf-4939-91fb-0a19ee0ea2bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a5c20d6425142189a83939254e5e3d1)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:17.307131 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f8e86f-82bf-4939-91fb-0a19ee0ea2bb admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:17.332988 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-8c87ae40-6496-48e2-a0c5-abe560b25882 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b080b84e286e4ceebdad70f530f23cce', 'request_id': 'req-8c87ae40-6496-48e2-a0c5-abe560b25882'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:17.341319 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-8c87ae40-6496-48e2-a0c5-abe560b25882 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b080b84e286e4ceebdad70f530f23cce', 'request_id': 'req-8c87ae40-6496-48e2-a0c5-abe560b25882'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:17.347810 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-8c87ae40-6496-48e2-a0c5-abe560b25882 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b080b84e286e4ceebdad70f530f23cce', 'request_id': 'req-8c87ae40-6496-48e2-a0c5-abe560b25882'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:17.358845 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-10f8e86f-82bf-4939-91fb-0a19ee0ea2bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a5c20d6425142189a83939254e5e3d1', 'request_id': 'req-10f8e86f-82bf-4939-91fb-0a19ee0ea2bb'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:17.361104 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-8c87ae40-6496-48e2-a0c5-abe560b25882 admin admin] The token cache is being invalidate because project b080b84e286e4ceebdad70f530f23cce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81661) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:22:17.361595 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-8c87ae40-6496-48e2-a0c5-abe560b25882 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:17.362413 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-8c87ae40-6496-48e2-a0c5-abe560b25882 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:17.363904 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1375/2762] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:22:17 2022] DELETE /identity/v3/projects/b080b84e286e4ceebdad70f530f23cce => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:17.365074 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-10f8e86f-82bf-4939-91fb-0a19ee0ea2bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a5c20d6425142189a83939254e5e3d1', 'request_id': 'req-10f8e86f-82bf-4939-91fb-0a19ee0ea2bb'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:17.368532 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c87ae40-6496-48e2-a0c5-abe560b25882 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:17.369875 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-8c87ae40-6496-48e2-a0c5-abe560b25882 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:17.371791 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-10f8e86f-82bf-4939-91fb-0a19ee0ea2bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a5c20d6425142189a83939254e5e3d1', 'request_id': 'req-10f8e86f-82bf-4939-91fb-0a19ee0ea2bb'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:17.374283 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:17.388942 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-10f8e86f-82bf-4939-91fb-0a19ee0ea2bb admin admin] The token cache is being invalidate because project 4a5c20d6425142189a83939254e5e3d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:22:17.389260 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-10f8e86f-82bf-4939-91fb-0a19ee0ea2bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:17.389929 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-10f8e86f-82bf-4939-91fb-0a19ee0ea2bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:17.391471 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1388/2763] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:22:17 2022] DELETE /identity/v3/projects/4a5c20d6425142189a83939254e5e3d1 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:17.395211 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f8e86f-82bf-4939-91fb-0a19ee0ea2bb admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:17.396644 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-10f8e86f-82bf-4939-91fb-0a19ee0ea2bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:17.435662 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c87ae40-6496-48e2-a0c5-abe560b25882 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:17.462070 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:17.464016 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f8e86f-82bf-4939-91fb-0a19ee0ea2bb admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:17.469489 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0cf37-a1a6-44ff-bda8-076bd98a8e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:17.470303 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0cf37-a1a6-44ff-bda8-076bd98a8e99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:17.470586 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0cf37-a1a6-44ff-bda8-076bd98a8e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:17.470856 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0cf37-a1a6-44ff-bda8-076bd98a8e99 admin admin] PATH_INFO: `/v3/users/e008a97d5c964934bd367835f3e4cb6a` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:17.471992 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d0cf37-a1a6-44ff-bda8-076bd98a8e99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e008a97d5c964934bd367835f3e4cb6a)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:17.473955 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d0cf37-a1a6-44ff-bda8-076bd98a8e99 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:17.495250 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f19c08d-7714-4bc0-a1c4-27ab410b1b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:17.496125 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f19c08d-7714-4bc0-a1c4-27ab410b1b39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:17.496441 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f19c08d-7714-4bc0-a1c4-27ab410b1b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:17.496714 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f19c08d-7714-4bc0-a1c4-27ab410b1b39 admin admin] PATH_INFO: `/v3/users/5ab192f84cca48a7a40415d45e2f949d` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:17.497909 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f19c08d-7714-4bc0-a1c4-27ab410b1b39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ab192f84cca48a7a40415d45e2f949d)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:17.500108 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f19c08d-7714-4bc0-a1c4-27ab410b1b39 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:17.524555 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-57d0cf37-a1a6-44ff-bda8-076bd98a8e99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e008a97d5c964934bd367835f3e4cb6a'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:17.538227 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-57d0cf37-a1a6-44ff-bda8-076bd98a8e99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e008a97d5c964934bd367835f3e4cb6a'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:17.546732 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-57d0cf37-a1a6-44ff-bda8-076bd98a8e99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e008a97d5c964934bd367835f3e4cb6a'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:17.550627 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-3f19c08d-7714-4bc0-a1c4-27ab410b1b39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ab192f84cca48a7a40415d45e2f949d'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:17.554735 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-57d0cf37-a1a6-44ff-bda8-076bd98a8e99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e008a97d5c964934bd367835f3e4cb6a'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:17.555400 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-57d0cf37-a1a6-44ff-bda8-076bd98a8e99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e008a97d5c964934bd367835f3e4cb6a'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:17.562536 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1389/2764] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:22:17 2022] DELETE /identity/v3/users/e008a97d5c964934bd367835f3e4cb6a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:22:17.565571 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-3f19c08d-7714-4bc0-a1c4-27ab410b1b39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ab192f84cca48a7a40415d45e2f949d'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:17.575863 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-3f19c08d-7714-4bc0-a1c4-27ab410b1b39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ab192f84cca48a7a40415d45e2f949d'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:17.576634 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-3f19c08d-7714-4bc0-a1c4-27ab410b1b39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ab192f84cca48a7a40415d45e2f949d'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:17.584975 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-3f19c08d-7714-4bc0-a1c4-27ab410b1b39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ab192f84cca48a7a40415d45e2f949d'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:17.591692 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1376/2765] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:22:17 2022] DELETE /identity/v3/users/5ab192f84cca48a7a40415d45e2f949d => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:22:17.857049 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0cf37-a1a6-44ff-bda8-076bd98a8e99 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:17.858956 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-57d0cf37-a1a6-44ff-bda8-076bd98a8e99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:17.934180 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0cf37-a1a6-44ff-bda8-076bd98a8e99 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:17.951887 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:17.995803 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1ea2e7-bac3-448c-9853-5732d7d19f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:17.996684 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1ea2e7-bac3-448c-9853-5732d7d19f08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:17.997119 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1ea2e7-bac3-448c-9853-5732d7d19f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:17.997615 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1ea2e7-bac3-448c-9853-5732d7d19f08 admin admin] PATH_INFO: `/v3/projects/0de0b244fa484769b3c823a776cd1ec3` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:17.999012 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1ea2e7-bac3-448c-9853-5732d7d19f08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0de0b244fa484769b3c823a776cd1ec3)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:18.000959 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1ea2e7-bac3-448c-9853-5732d7d19f08 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:18.051493 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-0f1ea2e7-bac3-448c-9853-5732d7d19f08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0de0b244fa484769b3c823a776cd1ec3', 'request_id': 'req-0f1ea2e7-bac3-448c-9853-5732d7d19f08'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:18.058474 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-0f1ea2e7-bac3-448c-9853-5732d7d19f08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0de0b244fa484769b3c823a776cd1ec3', 'request_id': 'req-0f1ea2e7-bac3-448c-9853-5732d7d19f08'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:18.064274 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-0f1ea2e7-bac3-448c-9853-5732d7d19f08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0de0b244fa484769b3c823a776cd1ec3', 'request_id': 'req-0f1ea2e7-bac3-448c-9853-5732d7d19f08'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:18.077523 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-0f1ea2e7-bac3-448c-9853-5732d7d19f08 admin admin] The token cache is being invalidate because project 0de0b244fa484769b3c823a776cd1ec3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:22:18.077818 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-0f1ea2e7-bac3-448c-9853-5732d7d19f08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:18.078444 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-0f1ea2e7-bac3-448c-9853-5732d7d19f08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:18.080114 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1390/2766] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:22:17 2022] DELETE /identity/v3/projects/0de0b244fa484769b3c823a776cd1ec3 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:18.083987 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f19c08d-7714-4bc0-a1c4-27ab410b1b39 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:18.086845 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-3f19c08d-7714-4bc0-a1c4-27ab410b1b39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:18.099641 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f19c08d-7714-4bc0-a1c4-27ab410b1b39 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:18.142135 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1ea2e7-bac3-448c-9853-5732d7d19f08 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:18.144128 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-0f1ea2e7-bac3-448c-9853-5732d7d19f08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:18.171121 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369d88f6-1a08-4814-a765-1ea8dbbf1d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:18.171763 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d88f6-1a08-4814-a765-1ea8dbbf1d46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:18.172066 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d88f6-1a08-4814-a765-1ea8dbbf1d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:18.172326 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d88f6-1a08-4814-a765-1ea8dbbf1d46 admin admin] PATH_INFO: `/v3/users/4f8c0d0925bc45aeb909f1fbf6320174` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:18.173446 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369d88f6-1a08-4814-a765-1ea8dbbf1d46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f8c0d0925bc45aeb909f1fbf6320174)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:18.175365 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369d88f6-1a08-4814-a765-1ea8dbbf1d46 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:18.221305 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1ea2e7-bac3-448c-9853-5732d7d19f08 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:18.226367 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5050a223-5f63-4f08-baca-3f3ba301a37c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:18.227138 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5050a223-5f63-4f08-baca-3f3ba301a37c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:18.227347 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5050a223-5f63-4f08-baca-3f3ba301a37c admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:18.227618 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5050a223-5f63-4f08-baca-3f3ba301a37c admin admin] PATH_INFO: `/v3/projects/09d2c84ae9ef457a9964b7549c3a2847` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:18.227978 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-369d88f6-1a08-4814-a765-1ea8dbbf1d46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f8c0d0925bc45aeb909f1fbf6320174'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:18.228770 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5050a223-5f63-4f08-baca-3f3ba301a37c admin admin] RBAC: Authorizing `identity:delete_project(project_id=09d2c84ae9ef457a9964b7549c3a2847)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:18.230336 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:18.230669 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5050a223-5f63-4f08-baca-3f3ba301a37c admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:18.244450 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-369d88f6-1a08-4814-a765-1ea8dbbf1d46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f8c0d0925bc45aeb909f1fbf6320174'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:18.251287 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-369d88f6-1a08-4814-a765-1ea8dbbf1d46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f8c0d0925bc45aeb909f1fbf6320174'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:18.251997 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-369d88f6-1a08-4814-a765-1ea8dbbf1d46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f8c0d0925bc45aeb909f1fbf6320174'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:18.260802 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-369d88f6-1a08-4814-a765-1ea8dbbf1d46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f8c0d0925bc45aeb909f1fbf6320174'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:18.267455 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1377/2767] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:22:18 2022] DELETE /identity/v3/users/4f8c0d0925bc45aeb909f1fbf6320174 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:18.280128 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-5050a223-5f63-4f08-baca-3f3ba301a37c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09d2c84ae9ef457a9964b7549c3a2847', 'request_id': 'req-5050a223-5f63-4f08-baca-3f3ba301a37c'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:18.282021 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:18.287100 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-5050a223-5f63-4f08-baca-3f3ba301a37c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09d2c84ae9ef457a9964b7549c3a2847', 'request_id': 'req-5050a223-5f63-4f08-baca-3f3ba301a37c'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:18.295756 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-5050a223-5f63-4f08-baca-3f3ba301a37c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09d2c84ae9ef457a9964b7549c3a2847', 'request_id': 'req-5050a223-5f63-4f08-baca-3f3ba301a37c'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:18.309918 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-5050a223-5f63-4f08-baca-3f3ba301a37c admin admin] The token cache is being invalidate because project 09d2c84ae9ef457a9964b7549c3a2847 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:22:18.310203 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-5050a223-5f63-4f08-baca-3f3ba301a37c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:18.310983 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-5050a223-5f63-4f08-baca-3f3ba301a37c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:18.312776 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1391/2768] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:22:18 2022] DELETE /identity/v3/projects/09d2c84ae9ef457a9964b7549c3a2847 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:18.317565 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38505f96-9455-4997-a61f-56c9e3a60431 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:18.317847 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38505f96-9455-4997-a61f-56c9e3a60431 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:18.318121 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38505f96-9455-4997-a61f-56c9e3a60431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:18.341570 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-38505f96-9455-4997-a61f-56c9e3a60431 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:18.342855 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-38505f96-9455-4997-a61f-56c9e3a60431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:18.409423 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1378/2769] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:22:18.412423 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e5771-63a7-4ff7-805f-56c68aaec195 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:18.412709 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e5771-63a7-4ff7-805f-56c68aaec195 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:18.413026 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e5771-63a7-4ff7-805f-56c68aaec195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:18.438367 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-6a0e5771-63a7-4ff7-805f-56c68aaec195 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:18.439581 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-6a0e5771-63a7-4ff7-805f-56c68aaec195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:18.448599 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1392/2770] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:22:18.453181 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38505f96-9455-4997-a61f-56c9e3a60431 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:18.463168 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38505f96-9455-4997-a61f-56c9e3a60431 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:18.467905 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acb13a3-9757-4774-9b9b-71297efe4dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:18.468634 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb13a3-9757-4774-9b9b-71297efe4dd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:18.468892 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb13a3-9757-4774-9b9b-71297efe4dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:18.469214 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb13a3-9757-4774-9b9b-71297efe4dd4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:18.469861 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acb13a3-9757-4774-9b9b-71297efe4dd4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:22:18.470163 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acb13a3-9757-4774-9b9b-71297efe4dd4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:18.471828 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acb13a3-9757-4774-9b9b-71297efe4dd4 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:18.484484 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1379/2771] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:22:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:22:18.488843 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e5771-63a7-4ff7-805f-56c68aaec195 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:18.498778 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e5771-63a7-4ff7-805f-56c68aaec195 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:18.503549 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9680cb8-f82e-4f5c-8296-e8b2342320ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:18.504198 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9680cb8-f82e-4f5c-8296-e8b2342320ad admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:18.504480 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9680cb8-f82e-4f5c-8296-e8b2342320ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:18.504754 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9680cb8-f82e-4f5c-8296-e8b2342320ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:18.505378 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9680cb8-f82e-4f5c-8296-e8b2342320ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:18.508084 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9680cb8-f82e-4f5c-8296-e8b2342320ad admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:18.536616 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1393/2772] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:22:18 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:22:18.541491 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acb13a3-9757-4774-9b9b-71297efe4dd4 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:18.551704 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9680cb8-f82e-4f5c-8296-e8b2342320ad admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:18.552248 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acb13a3-9757-4774-9b9b-71297efe4dd4 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:18.553388 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-d9680cb8-f82e-4f5c-8296-e8b2342320ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:18.613167 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9fd7ad-0048-4638-9dd6-0bed62b19092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:18.613777 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9fd7ad-0048-4638-9dd6-0bed62b19092 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:18.614057 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9fd7ad-0048-4638-9dd6-0bed62b19092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:18.614466 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9fd7ad-0048-4638-9dd6-0bed62b19092 admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:18.615126 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9fd7ad-0048-4638-9dd6-0bed62b19092 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:18.617729 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9fd7ad-0048-4638-9dd6-0bed62b19092 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:18.628187 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9680cb8-f82e-4f5c-8296-e8b2342320ad admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:18.633738 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455742db-94e1-4a1d-970d-d52251748e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:18.634513 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455742db-94e1-4a1d-970d-d52251748e31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:18.634855 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455742db-94e1-4a1d-970d-d52251748e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:18.635268 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455742db-94e1-4a1d-970d-d52251748e31 admin admin] PATH_INFO: `/v3/projects/17c2be1545a841b6a148c63f98ac7234` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:18.636510 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455742db-94e1-4a1d-970d-d52251748e31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17c2be1545a841b6a148c63f98ac7234)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:18.638573 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455742db-94e1-4a1d-970d-d52251748e31 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:18.646602 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1380/2773] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:22:18 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:22:18.647747 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9fd7ad-0048-4638-9dd6-0bed62b19092 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:18.658122 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9fd7ad-0048-4638-9dd6-0bed62b19092 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:18.662745 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff3188-acb2-4d9e-8287-4c57e29c6844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:18.663388 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff3188-acb2-4d9e-8287-4c57e29c6844 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:18.664329 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff3188-acb2-4d9e-8287-4c57e29c6844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:18.664597 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff3188-acb2-4d9e-8287-4c57e29c6844 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:18.665220 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ff3188-acb2-4d9e-8287-4c57e29c6844 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:18.667027 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ff3188-acb2-4d9e-8287-4c57e29c6844 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:18.679821 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1381/2774] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:22:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:22:18.683886 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff3188-acb2-4d9e-8287-4c57e29c6844 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:18.690022 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-455742db-94e1-4a1d-970d-d52251748e31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17c2be1545a841b6a148c63f98ac7234', 'request_id': 'req-455742db-94e1-4a1d-970d-d52251748e31'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:18.693710 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff3188-acb2-4d9e-8287-4c57e29c6844 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:18.696651 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-455742db-94e1-4a1d-970d-d52251748e31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17c2be1545a841b6a148c63f98ac7234', 'request_id': 'req-455742db-94e1-4a1d-970d-d52251748e31'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:18.702377 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-455742db-94e1-4a1d-970d-d52251748e31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17c2be1545a841b6a148c63f98ac7234', 'request_id': 'req-455742db-94e1-4a1d-970d-d52251748e31'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:18.716181 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-455742db-94e1-4a1d-970d-d52251748e31 admin admin] The token cache is being invalidate because project 17c2be1545a841b6a148c63f98ac7234 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:22:18.716474 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-455742db-94e1-4a1d-970d-d52251748e31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:18.717132 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-455742db-94e1-4a1d-970d-d52251748e31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:18.718725 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1394/2775] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:22:18 2022] DELETE /identity/v3/projects/17c2be1545a841b6a148c63f98ac7234 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:18.725208 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:18.725344 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8cb721-96b6-418d-b7b0-4347aa1d8ba4 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:18.725551 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8cb721-96b6-418d-b7b0-4347aa1d8ba4 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:18.725831 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8cb721-96b6-418d-b7b0-4347aa1d8ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:18.748929 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-3f8cb721-96b6-418d-b7b0-4347aa1d8ba4 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:18.750128 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-3f8cb721-96b6-418d-b7b0-4347aa1d8ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:18.754304 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba18dda-2f67-4f37-b6a5-36f1b4e52163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:18.754905 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba18dda-2f67-4f37-b6a5-36f1b4e52163 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:18.755183 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba18dda-2f67-4f37-b6a5-36f1b4e52163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:18.755452 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba18dda-2f67-4f37-b6a5-36f1b4e52163 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:18.756088 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba18dda-2f67-4f37-b6a5-36f1b4e52163 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:18.757741 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba18dda-2f67-4f37-b6a5-36f1b4e52163 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:18.758959 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1395/2776] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:22:18.771379 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1382/2777] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:22:18 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 01:22:18.771579 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c82569-c3a9-491f-acb5-3ac3d6d93993 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:18.771579 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c82569-c3a9-491f-acb5-3ac3d6d93993 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:18.771579 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c82569-c3a9-491f-acb5-3ac3d6d93993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:18.772382 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba18dda-2f67-4f37-b6a5-36f1b4e52163 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:18.774219 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-1ba18dda-2f67-4f37-b6a5-36f1b4e52163 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:18.786947 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba18dda-2f67-4f37-b6a5-36f1b4e52163 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:18.791334 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-d4c82569-c3a9-491f-acb5-3ac3d6d93993 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:18.792604 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-d4c82569-c3a9-491f-acb5-3ac3d6d93993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:18.793150 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6726d72-7e30-4673-a21f-4b1b80e89c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:18.793834 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6726d72-7e30-4673-a21f-4b1b80e89c46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:18.794113 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6726d72-7e30-4673-a21f-4b1b80e89c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:18.794392 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6726d72-7e30-4673-a21f-4b1b80e89c46 admin admin] PATH_INFO: `/v3/projects/09e167cc2d6d4f1bb5dd4b90c35ce02e/users/c922fa6d4cb446f5ae0dc16e7fe8d29d/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:18.800775 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1396/2778] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:22:18.805525 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c82569-c3a9-491f-acb5-3ac3d6d93993 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:18.810294 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6726d72-7e30-4673-a21f-4b1b80e89c46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09e167cc2d6d4f1bb5dd4b90c35ce02e, user_id=c922fa6d4cb446f5ae0dc16e7fe8d29d, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:18.813030 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6726d72-7e30-4673-a21f-4b1b80e89c46 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:18.814696 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c82569-c3a9-491f-acb5-3ac3d6d93993 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:18.820849 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9416761a-b22a-422c-b88b-4cccbbded3c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:18.821631 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416761a-b22a-422c-b88b-4cccbbded3c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:18.822302 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416761a-b22a-422c-b88b-4cccbbded3c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:18.822575 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416761a-b22a-422c-b88b-4cccbbded3c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:18.823340 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9416761a-b22a-422c-b88b-4cccbbded3c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:22:18.823655 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9416761a-b22a-422c-b88b-4cccbbded3c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:18.825278 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9416761a-b22a-422c-b88b-4cccbbded3c4 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:18.826342 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1383/2779] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:22:18 2022] PUT /identity/v3/projects/09e167cc2d6d4f1bb5dd4b90c35ce02e/users/c922fa6d4cb446f5ae0dc16e7fe8d29d/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:22:18.837036 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f97939-3dac-4546-85d4-fee136989283 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:18.837324 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f97939-3dac-4546-85d4-fee136989283 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:18.837598 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f97939-3dac-4546-85d4-fee136989283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:18.838635 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1397/2780] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:22:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 01:22:18.844101 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9416761a-b22a-422c-b88b-4cccbbded3c4 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:18.860990 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9416761a-b22a-422c-b88b-4cccbbded3c4 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:18.866183 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-c2f97939-3dac-4546-85d4-fee136989283 None None] MFA Rules not processed for user `c922fa6d4cb446f5ae0dc16e7fe8d29d`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:18.868013 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-c2f97939-3dac-4546-85d4-fee136989283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:18.877177 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:18.926536 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081f84e9-1e75-4a02-8d19-74d2dac1a47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:18.927243 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081f84e9-1e75-4a02-8d19-74d2dac1a47c admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:18.927623 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081f84e9-1e75-4a02-8d19-74d2dac1a47c admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:18.927798 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081f84e9-1e75-4a02-8d19-74d2dac1a47c admin admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:18.928488 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081f84e9-1e75-4a02-8d19-74d2dac1a47c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:18.930555 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081f84e9-1e75-4a02-8d19-74d2dac1a47c admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:18.935382 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1384/2781] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:22:18.947143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f97939-3dac-4546-85d4-fee136989283 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:18.948935 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-c2f97939-3dac-4546-85d4-fee136989283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:18.962609 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1398/2782] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:22:18 2022] POST /identity/v3/projects => generated 387 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:22:18.963712 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081f84e9-1e75-4a02-8d19-74d2dac1a47c admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:19.010664 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f97939-3dac-4546-85d4-fee136989283 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:19.027821 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081f84e9-1e75-4a02-8d19-74d2dac1a47c admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:19.032587 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec53dac4-6268-4a84-a52a-f5b7723565c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:19.033191 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec53dac4-6268-4a84-a52a-f5b7723565c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:19.033470 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec53dac4-6268-4a84-a52a-f5b7723565c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:19.033748 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec53dac4-6268-4a84-a52a-f5b7723565c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:19.034396 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec53dac4-6268-4a84-a52a-f5b7723565c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:19.036541 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec53dac4-6268-4a84-a52a-f5b7723565c1 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:19.060769 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1fd9ed-eab1-45bb-9ba3-a3448a612c3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:19.061412 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fd9ed-eab1-45bb-9ba3-a3448a612c3d service cinder] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:19.061730 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fd9ed-eab1-45bb-9ba3-a3448a612c3d service cinder] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:19.061979 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1399/2783] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:22:18 2022] POST /identity/v3/users => generated 294 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 01:22:19.064183 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fd9ed-eab1-45bb-9ba3-a3448a612c3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:19.066018 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec53dac4-6268-4a84-a52a-f5b7723565c1 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:19.068941 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:19.075968 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec53dac4-6268-4a84-a52a-f5b7723565c1 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:19.080709 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a891a6-2cc2-426c-9faa-bfbdee56f085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:19.081299 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a891a6-2cc2-426c-9faa-bfbdee56f085 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:19.081584 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a891a6-2cc2-426c-9faa-bfbdee56f085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:19.081862 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a891a6-2cc2-426c-9faa-bfbdee56f085 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:19.082498 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a891a6-2cc2-426c-9faa-bfbdee56f085 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:19.084086 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a891a6-2cc2-426c-9faa-bfbdee56f085 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:19.094963 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1400/2784] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:22:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 01:22:19.098746 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a891a6-2cc2-426c-9faa-bfbdee56f085 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:19.109031 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a891a6-2cc2-426c-9faa-bfbdee56f085 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:19.113697 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe083e3b-80da-49b0-87d7-b90d301e4e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:19.114279 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe083e3b-80da-49b0-87d7-b90d301e4e2d admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:19.114559 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe083e3b-80da-49b0-87d7-b90d301e4e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:19.114838 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe083e3b-80da-49b0-87d7-b90d301e4e2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:19.115497 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe083e3b-80da-49b0-87d7-b90d301e4e2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:19.117143 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe083e3b-80da-49b0-87d7-b90d301e4e2d admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:19.124994 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1fd9ed-eab1-45bb-9ba3-a3448a612c3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:19.126428 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1fd9ed-eab1-45bb-9ba3-a3448a612c3d service cinder] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:19.127639 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1401/2785] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:22:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 01:22:19.131636 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe083e3b-80da-49b0-87d7-b90d301e4e2d admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:19.136528 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1385/2786] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:22:18 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 01:22:19.143595 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe083e3b-80da-49b0-87d7-b90d301e4e2d admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:19.148817 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3127ed-3039-42b9-bb4b-f4fe0234e749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:19.149466 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3127ed-3039-42b9-bb4b-f4fe0234e749 admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:19.149735 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3127ed-3039-42b9-bb4b-f4fe0234e749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:19.150010 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3127ed-3039-42b9-bb4b-f4fe0234e749 admin admin] PATH_INFO: `/v3/projects/2d80aa1e0e3344d39e66c0ad1dadfad5/users/aabbbd6ec40d420ab490e757ced48f18/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:19.164837 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3127ed-3039-42b9-bb4b-f4fe0234e749 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d80aa1e0e3344d39e66c0ad1dadfad5, user_id=aabbbd6ec40d420ab490e757ced48f18, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:19.167527 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3127ed-3039-42b9-bb4b-f4fe0234e749 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:19.181058 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1402/2787] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:22:19 2022] PUT /identity/v3/projects/2d80aa1e0e3344d39e66c0ad1dadfad5/users/aabbbd6ec40d420ab490e757ced48f18/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:22:19.192310 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c33e7-9aca-4f38-8aca-f6821331c7aa None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:19.192673 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c33e7-9aca-4f38-8aca-f6821331c7aa None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:19.193039 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c33e7-9aca-4f38-8aca-f6821331c7aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:19.221740 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-849c33e7-9aca-4f38-8aca-f6821331c7aa None None] MFA Rules not processed for user `aabbbd6ec40d420ab490e757ced48f18`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:19.223132 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-849c33e7-9aca-4f38-8aca-f6821331c7aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:19.298035 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1386/2788] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:19 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:22:19.303737 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3127ed-3039-42b9-bb4b-f4fe0234e749 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:19.353795 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:19.373623 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3127ed-3039-42b9-bb4b-f4fe0234e749 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:19.378657 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b2a44c-60b4-420d-bd20-17e961fe4e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:19.379310 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b2a44c-60b4-420d-bd20-17e961fe4e84 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:19.379540 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b2a44c-60b4-420d-bd20-17e961fe4e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:19.379799 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b2a44c-60b4-420d-bd20-17e961fe4e84 admin admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:19.380457 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b2a44c-60b4-420d-bd20-17e961fe4e84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:19.383572 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b2a44c-60b4-420d-bd20-17e961fe4e84 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:19.415219 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1403/2789] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:22:19 2022] POST /identity/v3/projects => generated 389 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:22:19.419116 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c33e7-9aca-4f38-8aca-f6821331c7aa None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:19.431671 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c33e7-9aca-4f38-8aca-f6821331c7aa None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:19.492946 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ab75b-827a-49e8-b641-d861c2223246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:19.493559 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ab75b-827a-49e8-b641-d861c2223246 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:19.493840 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ab75b-827a-49e8-b641-d861c2223246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:19.494114 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ab75b-827a-49e8-b641-d861c2223246 admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:19.494751 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2ab75b-827a-49e8-b641-d861c2223246 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:19.496748 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2ab75b-827a-49e8-b641-d861c2223246 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:19.518413 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1387/2790] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:22:19 2022] POST /identity/v3/users => generated 301 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 01:22:19.522476 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b2a44c-60b4-420d-bd20-17e961fe4e84 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:19.533131 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b2a44c-60b4-420d-bd20-17e961fe4e84 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:19.537949 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bc4735-8874-4d7b-ae47-9c7c32987754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:19.539175 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc4735-8874-4d7b-ae47-9c7c32987754 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:19.539670 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc4735-8874-4d7b-ae47-9c7c32987754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:19.539975 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc4735-8874-4d7b-ae47-9c7c32987754 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:19.540639 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bc4735-8874-4d7b-ae47-9c7c32987754 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:19.542251 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bc4735-8874-4d7b-ae47-9c7c32987754 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:19.552737 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1404/2791] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:22:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:22:19.557293 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ab75b-827a-49e8-b641-d861c2223246 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:19.567197 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ab75b-827a-49e8-b641-d861c2223246 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:19.572193 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c22f16-032c-43fb-a685-5d9d7221563b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:19.573929 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c22f16-032c-43fb-a685-5d9d7221563b admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:19.573929 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c22f16-032c-43fb-a685-5d9d7221563b admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:19.573929 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c22f16-032c-43fb-a685-5d9d7221563b admin admin] PATH_INFO: `/v3/projects/286b077a364e4dcab7cd1b86201d955c/users/3fee47474e134928822775bfb92ef561/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:19.587912 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c22f16-032c-43fb-a685-5d9d7221563b admin admin] RBAC: Authorizing `identity:create_grant(project_id=286b077a364e4dcab7cd1b86201d955c, user_id=3fee47474e134928822775bfb92ef561, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:19.590619 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c22f16-032c-43fb-a685-5d9d7221563b admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:19.599097 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:19.604564 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1388/2792] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:22:19 2022] PUT /identity/v3/projects/286b077a364e4dcab7cd1b86201d955c/users/3fee47474e134928822775bfb92ef561/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:19.608497 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bc4735-8874-4d7b-ae47-9c7c32987754 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:19.619762 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bc4735-8874-4d7b-ae47-9c7c32987754 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:19.677850 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80671986-983a-4c38-a9cf-e369c57db6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:19.678457 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80671986-983a-4c38-a9cf-e369c57db6c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:19.678736 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80671986-983a-4c38-a9cf-e369c57db6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:19.679003 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80671986-983a-4c38-a9cf-e369c57db6c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:19.679629 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80671986-983a-4c38-a9cf-e369c57db6c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:19.681242 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80671986-983a-4c38-a9cf-e369c57db6c7 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:19.690806 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1405/2793] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:22:19 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:22:19.695403 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c22f16-032c-43fb-a685-5d9d7221563b admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:19.705678 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c22f16-032c-43fb-a685-5d9d7221563b admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:19.710999 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b09c20-5361-4da0-b8a6-275d120127dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:19.711768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b09c20-5361-4da0-b8a6-275d120127dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:19.712049 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b09c20-5361-4da0-b8a6-275d120127dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:19.712451 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b09c20-5361-4da0-b8a6-275d120127dc admin admin] PATH_INFO: `/v3/projects/286b077a364e4dcab7cd1b86201d955c/users/3fee47474e134928822775bfb92ef561/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:19.714358 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b09c20-5361-4da0-b8a6-275d120127dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=286b077a364e4dcab7cd1b86201d955c, user_id=3fee47474e134928822775bfb92ef561, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:19.717008 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b09c20-5361-4da0-b8a6-275d120127dc admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:19.727674 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1389/2794] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:22:19 2022] PUT /identity/v3/projects/286b077a364e4dcab7cd1b86201d955c/users/3fee47474e134928822775bfb92ef561/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:19.737969 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be50730d-cfbb-40dd-9e30-605fb0b725de None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:19.738250 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be50730d-cfbb-40dd-9e30-605fb0b725de None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:19.738601 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be50730d-cfbb-40dd-9e30-605fb0b725de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:19.766586 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-be50730d-cfbb-40dd-9e30-605fb0b725de None None] MFA Rules not processed for user `3fee47474e134928822775bfb92ef561`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:19.767781 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-be50730d-cfbb-40dd-9e30-605fb0b725de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:19.837333 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1406/2795] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:19 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:22:19.848500 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b09c20-5361-4da0-b8a6-275d120127dc admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:19.907073 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b09c20-5361-4da0-b8a6-275d120127dc admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:19.910776 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcc5832-f031-499f-a542-526807a7cc53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:19.911374 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc5832-f031-499f-a542-526807a7cc53 service cinder] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:19.911655 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc5832-f031-499f-a542-526807a7cc53 service cinder] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:19.911929 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc5832-f031-499f-a542-526807a7cc53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:19.921628 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcc5832-f031-499f-a542-526807a7cc53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:19.923070 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcc5832-f031-499f-a542-526807a7cc53 service cinder] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:19.932576 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1390/2796] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:22:19 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:22:20.068326 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be50730d-cfbb-40dd-9e30-605fb0b725de None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:20.111516 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:20.133629 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be50730d-cfbb-40dd-9e30-605fb0b725de None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:20.138257 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eab882e-d3bd-46aa-8d0f-1b9682104868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:20.138873 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab882e-d3bd-46aa-8d0f-1b9682104868 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:20.139171 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab882e-d3bd-46aa-8d0f-1b9682104868 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:20.139436 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab882e-d3bd-46aa-8d0f-1b9682104868 admin admin] PATH_INFO: `/v3/users/aabbbd6ec40d420ab490e757ced48f18` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:20.140569 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eab882e-d3bd-46aa-8d0f-1b9682104868 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aabbbd6ec40d420ab490e757ced48f18)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:20.142469 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eab882e-d3bd-46aa-8d0f-1b9682104868 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:20.189721 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-3eab882e-d3bd-46aa-8d0f-1b9682104868 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aabbbd6ec40d420ab490e757ced48f18'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:20.202363 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-3eab882e-d3bd-46aa-8d0f-1b9682104868 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aabbbd6ec40d420ab490e757ced48f18'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:20.209267 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-3eab882e-d3bd-46aa-8d0f-1b9682104868 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aabbbd6ec40d420ab490e757ced48f18'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:20.217519 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-3eab882e-d3bd-46aa-8d0f-1b9682104868 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aabbbd6ec40d420ab490e757ced48f18'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:20.218144 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-3eab882e-d3bd-46aa-8d0f-1b9682104868 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aabbbd6ec40d420ab490e757ced48f18'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:20.224114 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1407/2797] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:22:20 2022] DELETE /identity/v3/users/aabbbd6ec40d420ab490e757ced48f18 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:20.228895 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e30a483-f6ca-4434-ba0b-03443fbb6d31 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:20.229380 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e30a483-f6ca-4434-ba0b-03443fbb6d31 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:20.229493 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e30a483-f6ca-4434-ba0b-03443fbb6d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:20.248165 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-1e30a483-f6ca-4434-ba0b-03443fbb6d31 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:20.249384 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-1e30a483-f6ca-4434-ba0b-03443fbb6d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:20.310990 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1391/2798] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:22:20.321328 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eab882e-d3bd-46aa-8d0f-1b9682104868 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:20.322757 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-3eab882e-d3bd-46aa-8d0f-1b9682104868 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:20.381078 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eab882e-d3bd-46aa-8d0f-1b9682104868 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:20.385795 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79da10b-b346-44e0-98a5-e53ae43f9ac5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:20.386398 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79da10b-b346-44e0-98a5-e53ae43f9ac5 service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:20.386677 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79da10b-b346-44e0-98a5-e53ae43f9ac5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:20.386949 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79da10b-b346-44e0-98a5-e53ae43f9ac5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:20.395905 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79da10b-b346-44e0-98a5-e53ae43f9ac5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:20.397366 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79da10b-b346-44e0-98a5-e53ae43f9ac5 service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:20.406736 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1408/2799] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:22:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:22:20.702503 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e30a483-f6ca-4434-ba0b-03443fbb6d31 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:20.704165 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-1e30a483-f6ca-4434-ba0b-03443fbb6d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:20.719736 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e30a483-f6ca-4434-ba0b-03443fbb6d31 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:20.726271 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d0efeb-1451-466f-87f5-d97cdefacc92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:20.727035 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d0efeb-1451-466f-87f5-d97cdefacc92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:20.727220 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d0efeb-1451-466f-87f5-d97cdefacc92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:20.727582 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d0efeb-1451-466f-87f5-d97cdefacc92 admin admin] PATH_INFO: `/v3/projects/2d80aa1e0e3344d39e66c0ad1dadfad5` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:20.728775 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d0efeb-1451-466f-87f5-d97cdefacc92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d80aa1e0e3344d39e66c0ad1dadfad5)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:20.730735 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d0efeb-1451-466f-87f5-d97cdefacc92 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:20.778006 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-23d0efeb-1451-466f-87f5-d97cdefacc92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d80aa1e0e3344d39e66c0ad1dadfad5', 'request_id': 'req-23d0efeb-1451-466f-87f5-d97cdefacc92'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:20.785017 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-23d0efeb-1451-466f-87f5-d97cdefacc92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d80aa1e0e3344d39e66c0ad1dadfad5', 'request_id': 'req-23d0efeb-1451-466f-87f5-d97cdefacc92'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:20.791658 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-23d0efeb-1451-466f-87f5-d97cdefacc92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d80aa1e0e3344d39e66c0ad1dadfad5', 'request_id': 'req-23d0efeb-1451-466f-87f5-d97cdefacc92'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:20.802623 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-23d0efeb-1451-466f-87f5-d97cdefacc92 admin admin] The token cache is being invalidate because project 2d80aa1e0e3344d39e66c0ad1dadfad5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81661) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:22:20.802911 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-23d0efeb-1451-466f-87f5-d97cdefacc92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:20.803552 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-23d0efeb-1451-466f-87f5-d97cdefacc92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:20.805974 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1392/2800] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:22:20 2022] DELETE /identity/v3/projects/2d80aa1e0e3344d39e66c0ad1dadfad5 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:20.809728 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79da10b-b346-44e0-98a5-e53ae43f9ac5 service neutron] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:20.819289 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79da10b-b346-44e0-98a5-e53ae43f9ac5 service neutron] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:20.875689 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4c5a69-77c7-4349-bb92-3787e6589080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:20.876244 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4c5a69-77c7-4349-bb92-3787e6589080 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:20.876500 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4c5a69-77c7-4349-bb92-3787e6589080 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:20.876784 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4c5a69-77c7-4349-bb92-3787e6589080 admin admin] PATH_INFO: `/v3/users/3fee47474e134928822775bfb92ef561` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:20.877916 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4c5a69-77c7-4349-bb92-3787e6589080 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fee47474e134928822775bfb92ef561)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:20.880230 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4c5a69-77c7-4349-bb92-3787e6589080 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:20.882743 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:20.928121 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-1d4c5a69-77c7-4349-bb92-3787e6589080 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fee47474e134928822775bfb92ef561'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:20.940578 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-1d4c5a69-77c7-4349-bb92-3787e6589080 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fee47474e134928822775bfb92ef561'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:20.947667 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-1d4c5a69-77c7-4349-bb92-3787e6589080 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fee47474e134928822775bfb92ef561'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:20.954945 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-1d4c5a69-77c7-4349-bb92-3787e6589080 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fee47474e134928822775bfb92ef561'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:20.955598 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-1d4c5a69-77c7-4349-bb92-3787e6589080 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fee47474e134928822775bfb92ef561'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:20.961838 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1409/2801] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:22:20 2022] DELETE /identity/v3/users/3fee47474e134928822775bfb92ef561 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:21.241954 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d0efeb-1451-466f-87f5-d97cdefacc92 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:21.243653 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-23d0efeb-1451-466f-87f5-d97cdefacc92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:21.257855 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d0efeb-1451-466f-87f5-d97cdefacc92 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:21.267694 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:21.317367 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b8130f-9d6e-4711-a543-936c86c939e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:21.317991 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b8130f-9d6e-4711-a543-936c86c939e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:21.318270 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b8130f-9d6e-4711-a543-936c86c939e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:21.318541 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b8130f-9d6e-4711-a543-936c86c939e5 admin admin] PATH_INFO: `/v3/projects/286b077a364e4dcab7cd1b86201d955c` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:21.319636 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b8130f-9d6e-4711-a543-936c86c939e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=286b077a364e4dcab7cd1b86201d955c)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:21.321652 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b8130f-9d6e-4711-a543-936c86c939e5 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:21.367654 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-02b8130f-9d6e-4711-a543-936c86c939e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '286b077a364e4dcab7cd1b86201d955c', 'request_id': 'req-02b8130f-9d6e-4711-a543-936c86c939e5'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:21.374669 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-02b8130f-9d6e-4711-a543-936c86c939e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '286b077a364e4dcab7cd1b86201d955c', 'request_id': 'req-02b8130f-9d6e-4711-a543-936c86c939e5'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:21.380856 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-02b8130f-9d6e-4711-a543-936c86c939e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '286b077a364e4dcab7cd1b86201d955c', 'request_id': 'req-02b8130f-9d6e-4711-a543-936c86c939e5'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:21.391388 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-02b8130f-9d6e-4711-a543-936c86c939e5 admin admin] The token cache is being invalidate because project 286b077a364e4dcab7cd1b86201d955c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81661) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:22:21.391674 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-02b8130f-9d6e-4711-a543-936c86c939e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:21.392339 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-02b8130f-9d6e-4711-a543-936c86c939e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:21.393809 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1393/2802] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:22:21 2022] DELETE /identity/v3/projects/286b077a364e4dcab7cd1b86201d955c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:21.399770 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6654b9a-cece-442f-9680-224e4840c0a6 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:21.400057 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6654b9a-cece-442f-9680-224e4840c0a6 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:21.400362 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6654b9a-cece-442f-9680-224e4840c0a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:21.420626 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-a6654b9a-cece-442f-9680-224e4840c0a6 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:21.421830 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-a6654b9a-cece-442f-9680-224e4840c0a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:21.480694 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1410/2803] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:22:21.543820 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b8e97-8a70-4fff-8d1e-7ce467f740d4 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:21.544092 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b8e97-8a70-4fff-8d1e-7ce467f740d4 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:21.544509 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b8e97-8a70-4fff-8d1e-7ce467f740d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:21.564539 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-b68b8e97-8a70-4fff-8d1e-7ce467f740d4 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:21.565750 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-b68b8e97-8a70-4fff-8d1e-7ce467f740d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:21.574250 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1394/2804] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:22:21.578403 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6654b9a-cece-442f-9680-224e4840c0a6 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:21.588867 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6654b9a-cece-442f-9680-224e4840c0a6 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:21.593502 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e37f29-8ea0-4ff2-9f3e-bbb08a74f431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:21.594095 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e37f29-8ea0-4ff2-9f3e-bbb08a74f431 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:21.594374 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e37f29-8ea0-4ff2-9f3e-bbb08a74f431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:21.594648 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e37f29-8ea0-4ff2-9f3e-bbb08a74f431 admin admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:21.595318 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e37f29-8ea0-4ff2-9f3e-bbb08a74f431 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:22:21.595624 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e37f29-8ea0-4ff2-9f3e-bbb08a74f431 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:21.597247 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e37f29-8ea0-4ff2-9f3e-bbb08a74f431 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:21.608720 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1411/2805] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:22:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:22:21.612748 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68b8e97-8a70-4fff-8d1e-7ce467f740d4 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:21.622121 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68b8e97-8a70-4fff-8d1e-7ce467f740d4 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:21.626952 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14731d2d-e603-48ef-bc2e-386ffe7d4a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:21.627549 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14731d2d-e603-48ef-bc2e-386ffe7d4a90 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:21.627828 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14731d2d-e603-48ef-bc2e-386ffe7d4a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:21.628101 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14731d2d-e603-48ef-bc2e-386ffe7d4a90 admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:21.628758 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14731d2d-e603-48ef-bc2e-386ffe7d4a90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:21.630677 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14731d2d-e603-48ef-bc2e-386ffe7d4a90 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:21.658733 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1395/2806] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:22:21 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:22:21.662886 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e37f29-8ea0-4ff2-9f3e-bbb08a74f431 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:21.673417 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e37f29-8ea0-4ff2-9f3e-bbb08a74f431 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:21.731495 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de97dce-84a9-42ed-8626-1251553affbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:21.732103 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de97dce-84a9-42ed-8626-1251553affbc admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:21.732405 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de97dce-84a9-42ed-8626-1251553affbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:21.732676 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de97dce-84a9-42ed-8626-1251553affbc admin admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:21.733312 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de97dce-84a9-42ed-8626-1251553affbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:21.735248 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de97dce-84a9-42ed-8626-1251553affbc admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:21.750988 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:21.757142 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1412/2807] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:22:21 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:22:21.762513 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14731d2d-e603-48ef-bc2e-386ffe7d4a90 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:21.772700 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14731d2d-e603-48ef-bc2e-386ffe7d4a90 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:21.777985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f5b359-e084-4424-993a-97ba058a2736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:21.778582 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5b359-e084-4424-993a-97ba058a2736 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:21.778863 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5b359-e084-4424-993a-97ba058a2736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:21.779136 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5b359-e084-4424-993a-97ba058a2736 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:21.779772 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f5b359-e084-4424-993a-97ba058a2736 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:21.781408 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f5b359-e084-4424-993a-97ba058a2736 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:21.791168 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1396/2808] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:22:21.794985 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de97dce-84a9-42ed-8626-1251553affbc admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:21.805367 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de97dce-84a9-42ed-8626-1251553affbc admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:21.809919 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbed5fcf-0ff3-4bda-9fcd-22d5fa531269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:21.810497 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbed5fcf-0ff3-4bda-9fcd-22d5fa531269 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:21.810764 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbed5fcf-0ff3-4bda-9fcd-22d5fa531269 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:21.811031 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbed5fcf-0ff3-4bda-9fcd-22d5fa531269 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:21.811693 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbed5fcf-0ff3-4bda-9fcd-22d5fa531269 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:21.813298 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbed5fcf-0ff3-4bda-9fcd-22d5fa531269 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:21.824829 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1413/2809] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 01:22:21.829054 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f5b359-e084-4424-993a-97ba058a2736 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:21.839847 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f5b359-e084-4424-993a-97ba058a2736 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:21.844549 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bb7998-472e-4606-9915-0d32fe9fe502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:21.845220 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb7998-472e-4606-9915-0d32fe9fe502 admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:21.845500 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb7998-472e-4606-9915-0d32fe9fe502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:21.845773 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb7998-472e-4606-9915-0d32fe9fe502 admin admin] PATH_INFO: `/v3/projects/921c606e2d594a768ef7419bd69b599f/users/4ed12c3ff0ea4bb3b74e970d349d3bd4/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:21.860122 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bb7998-472e-4606-9915-0d32fe9fe502 admin admin] RBAC: Authorizing `identity:create_grant(project_id=921c606e2d594a768ef7419bd69b599f, user_id=4ed12c3ff0ea4bb3b74e970d349d3bd4, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:21.862796 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bb7998-472e-4606-9915-0d32fe9fe502 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:21.875617 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1397/2810] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:22:21 2022] PUT /identity/v3/projects/921c606e2d594a768ef7419bd69b599f/users/4ed12c3ff0ea4bb3b74e970d349d3bd4/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:21.886446 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1affdd5-bf9d-42fa-b298-8fcbd44316bd None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:21.886667 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1affdd5-bf9d-42fa-b298-8fcbd44316bd None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:21.886934 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1affdd5-bf9d-42fa-b298-8fcbd44316bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:21.915625 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-a1affdd5-bf9d-42fa-b298-8fcbd44316bd None None] MFA Rules not processed for user `4ed12c3ff0ea4bb3b74e970d349d3bd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:21.916851 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-a1affdd5-bf9d-42fa-b298-8fcbd44316bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:21.985509 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1414/2811] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:22:21.997800 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bb7998-472e-4606-9915-0d32fe9fe502 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:21.999616 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-11bb7998-472e-4606-9915-0d32fe9fe502 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:22.043177 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:22.059103 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bb7998-472e-4606-9915-0d32fe9fe502 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:22.062715 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c12fb0-d6c8-4bf4-9cb9-f07e191c6509 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1acfa7a09d34e2a9045c3f3fbea0f52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:22.063340 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c12fb0-d6c8-4bf4-9cb9-f07e191c6509 service glance] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:22.063614 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c12fb0-d6c8-4bf4-9cb9-f07e191c6509 service glance] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:22.063883 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c12fb0-d6c8-4bf4-9cb9-f07e191c6509 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:22.072824 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c12fb0-d6c8-4bf4-9cb9-f07e191c6509 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:22.074285 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c12fb0-d6c8-4bf4-9cb9-f07e191c6509 service glance] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:22.082832 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1398/2812] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:22:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 01:22:26.556862 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1affdd5-bf9d-42fa-b298-8fcbd44316bd None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:26.558539 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-a1affdd5-bf9d-42fa-b298-8fcbd44316bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:26.624074 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1affdd5-bf9d-42fa-b298-8fcbd44316bd None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:26.628651 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb900f6-62f0-400b-a4f8-58a45bdfda88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:26.629325 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb900f6-62f0-400b-a4f8-58a45bdfda88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:26.629606 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb900f6-62f0-400b-a4f8-58a45bdfda88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:26.629881 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb900f6-62f0-400b-a4f8-58a45bdfda88 admin admin] PATH_INFO: `/v3/users/c922fa6d4cb446f5ae0dc16e7fe8d29d` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:26.631058 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb900f6-62f0-400b-a4f8-58a45bdfda88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c922fa6d4cb446f5ae0dc16e7fe8d29d)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:26.633002 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb900f6-62f0-400b-a4f8-58a45bdfda88 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:26.680741 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-7bb900f6-62f0-400b-a4f8-58a45bdfda88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c922fa6d4cb446f5ae0dc16e7fe8d29d'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:26.693553 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-7bb900f6-62f0-400b-a4f8-58a45bdfda88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c922fa6d4cb446f5ae0dc16e7fe8d29d'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:26.707073 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-7bb900f6-62f0-400b-a4f8-58a45bdfda88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c922fa6d4cb446f5ae0dc16e7fe8d29d'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:26.711551 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:26.716657 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-7bb900f6-62f0-400b-a4f8-58a45bdfda88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c922fa6d4cb446f5ae0dc16e7fe8d29d'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:26.717358 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-7bb900f6-62f0-400b-a4f8-58a45bdfda88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c922fa6d4cb446f5ae0dc16e7fe8d29d'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:26.725969 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1415/2813] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:22:26 2022] DELETE /identity/v3/users/c922fa6d4cb446f5ae0dc16e7fe8d29d => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:26.730847 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a48a46-5ae8-41e7-a151-6bcc888e150c None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:26.731342 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a48a46-5ae8-41e7-a151-6bcc888e150c None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:26.731502 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a48a46-5ae8-41e7-a151-6bcc888e150c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:26.754340 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-41a48a46-5ae8-41e7-a151-6bcc888e150c None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:26.755559 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-41a48a46-5ae8-41e7-a151-6bcc888e150c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:26.816197 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1399/2814] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:22:26.826627 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb900f6-62f0-400b-a4f8-58a45bdfda88 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:26.828021 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-7bb900f6-62f0-400b-a4f8-58a45bdfda88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:26.891892 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb900f6-62f0-400b-a4f8-58a45bdfda88 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:26.895811 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f27b5e-a8fb-4941-bbde-ce6f086d90ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:26.896534 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f27b5e-a8fb-4941-bbde-ce6f086d90ea service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:26.896813 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f27b5e-a8fb-4941-bbde-ce6f086d90ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:26.897085 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f27b5e-a8fb-4941-bbde-ce6f086d90ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:26.907089 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f27b5e-a8fb-4941-bbde-ce6f086d90ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:26.908611 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f27b5e-a8fb-4941-bbde-ce6f086d90ea service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:26.918280 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1416/2815] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:22:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:22:27.204746 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a48a46-5ae8-41e7-a151-6bcc888e150c None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:27.206621 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-41a48a46-5ae8-41e7-a151-6bcc888e150c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:27.222915 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a48a46-5ae8-41e7-a151-6bcc888e150c None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:27.228399 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9e89aa-8489-49d0-86a6-65f985f631e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:27.229711 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e89aa-8489-49d0-86a6-65f985f631e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:27.230244 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e89aa-8489-49d0-86a6-65f985f631e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:27.230634 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e89aa-8489-49d0-86a6-65f985f631e7 admin admin] PATH_INFO: `/v3/projects/09e167cc2d6d4f1bb5dd4b90c35ce02e` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:27.232166 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9e89aa-8489-49d0-86a6-65f985f631e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09e167cc2d6d4f1bb5dd4b90c35ce02e)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:27.235470 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9e89aa-8489-49d0-86a6-65f985f631e7 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:27.288736 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-6d9e89aa-8489-49d0-86a6-65f985f631e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09e167cc2d6d4f1bb5dd4b90c35ce02e', 'request_id': 'req-6d9e89aa-8489-49d0-86a6-65f985f631e7'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:27.301774 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-6d9e89aa-8489-49d0-86a6-65f985f631e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09e167cc2d6d4f1bb5dd4b90c35ce02e', 'request_id': 'req-6d9e89aa-8489-49d0-86a6-65f985f631e7'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:27.305282 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-6d9e89aa-8489-49d0-86a6-65f985f631e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09e167cc2d6d4f1bb5dd4b90c35ce02e', 'request_id': 'req-6d9e89aa-8489-49d0-86a6-65f985f631e7'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:27.316023 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-6d9e89aa-8489-49d0-86a6-65f985f631e7 admin admin] The token cache is being invalidate because project 09e167cc2d6d4f1bb5dd4b90c35ce02e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81661) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:22:27.316344 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-6d9e89aa-8489-49d0-86a6-65f985f631e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:27.317012 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-6d9e89aa-8489-49d0-86a6-65f985f631e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:27.318566 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1400/2816] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:22:27 2022] DELETE /identity/v3/projects/09e167cc2d6d4f1bb5dd4b90c35ce02e => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:27.323738 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c6131-b299-4763-873e-67eec704d08f None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:27.324014 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c6131-b299-4763-873e-67eec704d08f None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:27.324312 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c6131-b299-4763-873e-67eec704d08f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:27.344203 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-820c6131-b299-4763-873e-67eec704d08f None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:27.345381 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-820c6131-b299-4763-873e-67eec704d08f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:27.406903 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1417/2817] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:22:27.470809 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d951038-5841-466e-88c1-d2beff20f079 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:27.471089 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d951038-5841-466e-88c1-d2beff20f079 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:27.471380 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d951038-5841-466e-88c1-d2beff20f079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:27.491573 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-9d951038-5841-466e-88c1-d2beff20f079 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:27.492869 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-9d951038-5841-466e-88c1-d2beff20f079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:27.501516 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1401/2818] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:22:27.505717 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c6131-b299-4763-873e-67eec704d08f None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:27.517170 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c6131-b299-4763-873e-67eec704d08f None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:27.522228 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae1478-1c1d-4cb4-893b-db05c3a8d379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:27.522868 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae1478-1c1d-4cb4-893b-db05c3a8d379 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:27.523167 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae1478-1c1d-4cb4-893b-db05c3a8d379 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:27.523439 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae1478-1c1d-4cb4-893b-db05c3a8d379 admin admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:27.524088 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eae1478-1c1d-4cb4-893b-db05c3a8d379 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:22:27.524422 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eae1478-1c1d-4cb4-893b-db05c3a8d379 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:27.526010 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eae1478-1c1d-4cb4-893b-db05c3a8d379 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:27.538377 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1418/2819] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:22:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:22:27.542780 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d951038-5841-466e-88c1-d2beff20f079 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:27.551968 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d951038-5841-466e-88c1-d2beff20f079 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:27.556703 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25980806-da20-45d8-a200-cac0ec3a0eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:27.557296 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25980806-da20-45d8-a200-cac0ec3a0eee admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:27.558103 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25980806-da20-45d8-a200-cac0ec3a0eee admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:27.558352 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25980806-da20-45d8-a200-cac0ec3a0eee admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:27.558972 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25980806-da20-45d8-a200-cac0ec3a0eee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:27.560998 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25980806-da20-45d8-a200-cac0ec3a0eee admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:27.590437 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1402/2820] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:22:27 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:22:27.594621 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae1478-1c1d-4cb4-893b-db05c3a8d379 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:27.605193 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae1478-1c1d-4cb4-893b-db05c3a8d379 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:27.663695 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7614a8-c12a-4d45-8af5-32bf91616919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:27.664331 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7614a8-c12a-4d45-8af5-32bf91616919 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:27.664691 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7614a8-c12a-4d45-8af5-32bf91616919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:27.665004 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7614a8-c12a-4d45-8af5-32bf91616919 admin admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:27.665898 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7614a8-c12a-4d45-8af5-32bf91616919 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:27.668479 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7614a8-c12a-4d45-8af5-32bf91616919 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:27.692596 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1419/2821] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:22:27 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:22:27.694090 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25980806-da20-45d8-a200-cac0ec3a0eee admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:27.704483 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25980806-da20-45d8-a200-cac0ec3a0eee admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:27.709408 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293ce27f-4b7c-4dcb-9535-a6f268f71985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:27.710025 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ce27f-4b7c-4dcb-9535-a6f268f71985 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:27.710304 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ce27f-4b7c-4dcb-9535-a6f268f71985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:27.710574 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ce27f-4b7c-4dcb-9535-a6f268f71985 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:27.711210 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293ce27f-4b7c-4dcb-9535-a6f268f71985 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:27.712853 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293ce27f-4b7c-4dcb-9535-a6f268f71985 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:27.723934 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1403/2822] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:22:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:22:27.727793 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7614a8-c12a-4d45-8af5-32bf91616919 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:27.738646 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7614a8-c12a-4d45-8af5-32bf91616919 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:27.743249 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d26811-f24c-4f2d-8ac3-95923a13bf7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:27.743841 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d26811-f24c-4f2d-8ac3-95923a13bf7b admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:27.744118 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d26811-f24c-4f2d-8ac3-95923a13bf7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:27.744420 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d26811-f24c-4f2d-8ac3-95923a13bf7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:27.745424 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d26811-f24c-4f2d-8ac3-95923a13bf7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:27.747069 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d26811-f24c-4f2d-8ac3-95923a13bf7b admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:27.757763 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1420/2823] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:22:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:22:27.762363 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293ce27f-4b7c-4dcb-9535-a6f268f71985 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:27.780311 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293ce27f-4b7c-4dcb-9535-a6f268f71985 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:27.785495 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797f675e-f54a-484a-84c7-5a4f3ca33705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:27.786345 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f675e-f54a-484a-84c7-5a4f3ca33705 admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:27.786688 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f675e-f54a-484a-84c7-5a4f3ca33705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:27.787024 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f675e-f54a-484a-84c7-5a4f3ca33705 admin admin] PATH_INFO: `/v3/projects/b5b86d965f7940238070c16f8760dc2d/users/4492641ab8054b4bba9d634dd54722c0/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:27.804808 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797f675e-f54a-484a-84c7-5a4f3ca33705 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5b86d965f7940238070c16f8760dc2d, user_id=4492641ab8054b4bba9d634dd54722c0, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:27.807744 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797f675e-f54a-484a-84c7-5a4f3ca33705 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:27.824389 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1404/2824] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:22:27 2022] PUT /identity/v3/projects/b5b86d965f7940238070c16f8760dc2d/users/4492641ab8054b4bba9d634dd54722c0/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:27.831739 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43875d5-dd55-4850-b198-361a953a458a None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:27.832027 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43875d5-dd55-4850-b198-361a953a458a None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:27.832332 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43875d5-dd55-4850-b198-361a953a458a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:27.841944 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:27.862798 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-d43875d5-dd55-4850-b198-361a953a458a None None] MFA Rules not processed for user `4492641ab8054b4bba9d634dd54722c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:27.864010 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-d43875d5-dd55-4850-b198-361a953a458a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:27.931501 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1421/2825] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:22:27.936374 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797f675e-f54a-484a-84c7-5a4f3ca33705 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:27.940773 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:28.001592 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797f675e-f54a-484a-84c7-5a4f3ca33705 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:28.006335 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176f744e-0a06-425c-8abd-46472f37b481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:28.006944 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176f744e-0a06-425c-8abd-46472f37b481 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:28.007228 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176f744e-0a06-425c-8abd-46472f37b481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:28.007501 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176f744e-0a06-425c-8abd-46472f37b481 admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:28.008455 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176f744e-0a06-425c-8abd-46472f37b481 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:28.011143 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176f744e-0a06-425c-8abd-46472f37b481 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:28.037486 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1405/2826] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:22:27 2022] POST /identity/v3/projects => generated 373 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:22:28.041607 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43875d5-dd55-4850-b198-361a953a458a None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:28.053485 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43875d5-dd55-4850-b198-361a953a458a None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:28.116229 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f0183-2077-49a8-98c4-4a93c5395331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:28.116890 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f0183-2077-49a8-98c4-4a93c5395331 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:28.117166 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f0183-2077-49a8-98c4-4a93c5395331 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:28.117436 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f0183-2077-49a8-98c4-4a93c5395331 admin admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:28.118115 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432f0183-2077-49a8-98c4-4a93c5395331 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:28.120082 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432f0183-2077-49a8-98c4-4a93c5395331 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:28.142847 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1422/2827] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:22:28 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:22:28.146848 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176f744e-0a06-425c-8abd-46472f37b481 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:28.157890 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176f744e-0a06-425c-8abd-46472f37b481 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:28.163961 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e8752d-960a-4a17-b14a-c022da18de24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:28.165013 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e8752d-960a-4a17-b14a-c022da18de24 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:28.165398 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e8752d-960a-4a17-b14a-c022da18de24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:28.165968 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e8752d-960a-4a17-b14a-c022da18de24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:28.166932 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e8752d-960a-4a17-b14a-c022da18de24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:28.169553 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e8752d-960a-4a17-b14a-c022da18de24 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:28.183386 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1406/2828] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:22:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:22:28.187590 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f0183-2077-49a8-98c4-4a93c5395331 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:28.197701 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f0183-2077-49a8-98c4-4a93c5395331 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:28.203127 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdd72a3-004a-4f4d-aeba-fad7f8e34147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:28.203778 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd72a3-004a-4f4d-aeba-fad7f8e34147 admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:28.204055 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd72a3-004a-4f4d-aeba-fad7f8e34147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:28.204375 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd72a3-004a-4f4d-aeba-fad7f8e34147 admin admin] PATH_INFO: `/v3/projects/aa8bbfee5f424d94bc84fd0457d81ddf/users/e0e548f03c6847e385930f775121d81b/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:28.221325 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdd72a3-004a-4f4d-aeba-fad7f8e34147 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa8bbfee5f424d94bc84fd0457d81ddf, user_id=e0e548f03c6847e385930f775121d81b, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:28.224427 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdd72a3-004a-4f4d-aeba-fad7f8e34147 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:28.238823 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1423/2829] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:22:28 2022] PUT /identity/v3/projects/aa8bbfee5f424d94bc84fd0457d81ddf/users/e0e548f03c6847e385930f775121d81b/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:28.242833 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e8752d-960a-4a17-b14a-c022da18de24 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:28.253750 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e8752d-960a-4a17-b14a-c022da18de24 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:28.314497 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677be071-fefb-435a-a1bd-3e27ac34a317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:28.315222 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677be071-fefb-435a-a1bd-3e27ac34a317 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:28.315497 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677be071-fefb-435a-a1bd-3e27ac34a317 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:28.316060 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677be071-fefb-435a-a1bd-3e27ac34a317 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:28.316725 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677be071-fefb-435a-a1bd-3e27ac34a317 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:28.318659 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677be071-fefb-435a-a1bd-3e27ac34a317 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:28.330360 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1407/2830] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:22:28 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:22:28.334435 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdd72a3-004a-4f4d-aeba-fad7f8e34147 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:28.344912 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdd72a3-004a-4f4d-aeba-fad7f8e34147 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:28.349523 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749a8887-0a66-4ed8-9c9a-1062e220dbfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:28.350157 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a8887-0a66-4ed8-9c9a-1062e220dbfe admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:28.350434 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a8887-0a66-4ed8-9c9a-1062e220dbfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:28.350704 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a8887-0a66-4ed8-9c9a-1062e220dbfe admin admin] PATH_INFO: `/v3/projects/aa8bbfee5f424d94bc84fd0457d81ddf/users/e0e548f03c6847e385930f775121d81b/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:28.352631 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749a8887-0a66-4ed8-9c9a-1062e220dbfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa8bbfee5f424d94bc84fd0457d81ddf, user_id=e0e548f03c6847e385930f775121d81b, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:28.355577 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749a8887-0a66-4ed8-9c9a-1062e220dbfe admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:28.366929 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1424/2831] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:22:28 2022] PUT /identity/v3/projects/aa8bbfee5f424d94bc84fd0457d81ddf/users/e0e548f03c6847e385930f775121d81b/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:28.377689 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f479999-3b5c-4163-aac1-af5ad96d8e36 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:28.377974 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f479999-3b5c-4163-aac1-af5ad96d8e36 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:28.378246 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f479999-3b5c-4163-aac1-af5ad96d8e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:28.407318 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-3f479999-3b5c-4163-aac1-af5ad96d8e36 None None] MFA Rules not processed for user `e0e548f03c6847e385930f775121d81b`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:28.408713 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-3f479999-3b5c-4163-aac1-af5ad96d8e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:28.473296 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:28.479695 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1408/2832] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:28 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:22:28.490514 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749a8887-0a66-4ed8-9c9a-1062e220dbfe admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:28.492240 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-749a8887-0a66-4ed8-9c9a-1062e220dbfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:28.548044 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:28.554088 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749a8887-0a66-4ed8-9c9a-1062e220dbfe admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:28.559008 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1251a0-6dc9-4f4c-bcc3-35b93a767492 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:28.559608 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1251a0-6dc9-4f4c-bcc3-35b93a767492 service cinder] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:28.559883 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1251a0-6dc9-4f4c-bcc3-35b93a767492 service cinder] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:28.560180 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1251a0-6dc9-4f4c-bcc3-35b93a767492 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:28.619312 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1251a0-6dc9-4f4c-bcc3-35b93a767492 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:28.621937 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1251a0-6dc9-4f4c-bcc3-35b93a767492 service cinder] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:28.632401 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1425/2833] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:22:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:22:31.212177 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f479999-3b5c-4163-aac1-af5ad96d8e36 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:31.275565 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f479999-3b5c-4163-aac1-af5ad96d8e36 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:31.280991 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f7eb8-3565-4a7a-9f2d-0beffab994dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:31.281621 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f7eb8-3565-4a7a-9f2d-0beffab994dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:31.281897 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f7eb8-3565-4a7a-9f2d-0beffab994dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:31.282165 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f7eb8-3565-4a7a-9f2d-0beffab994dc admin admin] PATH_INFO: `/v3/users/4492641ab8054b4bba9d634dd54722c0` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:31.283263 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7f7eb8-3565-4a7a-9f2d-0beffab994dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=4492641ab8054b4bba9d634dd54722c0)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:31.285769 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7f7eb8-3565-4a7a-9f2d-0beffab994dc admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:31.343534 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-0e7f7eb8-3565-4a7a-9f2d-0beffab994dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4492641ab8054b4bba9d634dd54722c0'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:31.356585 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-0e7f7eb8-3565-4a7a-9f2d-0beffab994dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4492641ab8054b4bba9d634dd54722c0'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:31.364011 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-0e7f7eb8-3565-4a7a-9f2d-0beffab994dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4492641ab8054b4bba9d634dd54722c0'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:31.364775 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-0e7f7eb8-3565-4a7a-9f2d-0beffab994dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4492641ab8054b4bba9d634dd54722c0'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:31.373098 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-0e7f7eb8-3565-4a7a-9f2d-0beffab994dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4492641ab8054b4bba9d634dd54722c0'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:31.379375 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1409/2834] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:22:31 2022] DELETE /identity/v3/users/4492641ab8054b4bba9d634dd54722c0 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:31.384705 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b51f3-0c78-4468-99df-1f8fc635f701 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:31.385203 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b51f3-0c78-4468-99df-1f8fc635f701 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:31.386591 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b51f3-0c78-4468-99df-1f8fc635f701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:31.406725 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-864b51f3-0c78-4468-99df-1f8fc635f701 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:31.407961 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-864b51f3-0c78-4468-99df-1f8fc635f701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:31.470621 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1426/2835] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:22:31.481127 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f7eb8-3565-4a7a-9f2d-0beffab994dc admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:31.482541 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-0e7f7eb8-3565-4a7a-9f2d-0beffab994dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:31.539752 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f7eb8-3565-4a7a-9f2d-0beffab994dc admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:31.544198 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e00121c-77d3-4d9c-9891-98e1c4537027 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:31.544796 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e00121c-77d3-4d9c-9891-98e1c4537027 service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:31.545074 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e00121c-77d3-4d9c-9891-98e1c4537027 service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:31.545343 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e00121c-77d3-4d9c-9891-98e1c4537027 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:31.555624 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e00121c-77d3-4d9c-9891-98e1c4537027 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:31.557158 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e00121c-77d3-4d9c-9891-98e1c4537027 service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:31.567010 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1410/2836] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:22:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 01:22:31.856788 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864b51f3-0c78-4468-99df-1f8fc635f701 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:31.858278 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-864b51f3-0c78-4468-99df-1f8fc635f701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:31.877670 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864b51f3-0c78-4468-99df-1f8fc635f701 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:31.886039 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f5721-8c72-41c6-a3e9-88508260eced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:31.886601 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f5721-8c72-41c6-a3e9-88508260eced admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:31.887065 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f5721-8c72-41c6-a3e9-88508260eced admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:31.887236 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f5721-8c72-41c6-a3e9-88508260eced admin admin] PATH_INFO: `/v3/projects/b5b86d965f7940238070c16f8760dc2d` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:31.888415 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120f5721-8c72-41c6-a3e9-88508260eced admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5b86d965f7940238070c16f8760dc2d)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:31.890430 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120f5721-8c72-41c6-a3e9-88508260eced admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:31.940195 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-120f5721-8c72-41c6-a3e9-88508260eced admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5b86d965f7940238070c16f8760dc2d', 'request_id': 'req-120f5721-8c72-41c6-a3e9-88508260eced'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:31.946665 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-120f5721-8c72-41c6-a3e9-88508260eced admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5b86d965f7940238070c16f8760dc2d', 'request_id': 'req-120f5721-8c72-41c6-a3e9-88508260eced'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:31.953241 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-120f5721-8c72-41c6-a3e9-88508260eced admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5b86d965f7940238070c16f8760dc2d', 'request_id': 'req-120f5721-8c72-41c6-a3e9-88508260eced'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:31.965789 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-120f5721-8c72-41c6-a3e9-88508260eced admin admin] The token cache is being invalidate because project b5b86d965f7940238070c16f8760dc2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:22:31.966078 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-120f5721-8c72-41c6-a3e9-88508260eced admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:31.966707 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-120f5721-8c72-41c6-a3e9-88508260eced admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:31.968243 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1427/2837] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:22:31 2022] DELETE /identity/v3/projects/b5b86d965f7940238070c16f8760dc2d => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:31.972389 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e00121c-77d3-4d9c-9891-98e1c4537027 service neutron] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:31.982687 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e00121c-77d3-4d9c-9891-98e1c4537027 service neutron] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:32.000531 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:32.054254 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c75dac-e54c-4d65-af79-ec08b7cf50c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:32.054893 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c75dac-e54c-4d65-af79-ec08b7cf50c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:32.055177 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c75dac-e54c-4d65-af79-ec08b7cf50c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:32.055451 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c75dac-e54c-4d65-af79-ec08b7cf50c1 admin admin] PATH_INFO: `/v3/users/e0e548f03c6847e385930f775121d81b` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:32.056655 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c75dac-e54c-4d65-af79-ec08b7cf50c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0e548f03c6847e385930f775121d81b)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:32.059323 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c75dac-e54c-4d65-af79-ec08b7cf50c1 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:32.106923 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-54c75dac-e54c-4d65-af79-ec08b7cf50c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0e548f03c6847e385930f775121d81b'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:32.118584 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-54c75dac-e54c-4d65-af79-ec08b7cf50c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0e548f03c6847e385930f775121d81b'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:32.126041 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-54c75dac-e54c-4d65-af79-ec08b7cf50c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0e548f03c6847e385930f775121d81b'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:32.126722 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-54c75dac-e54c-4d65-af79-ec08b7cf50c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0e548f03c6847e385930f775121d81b'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:32.134331 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-54c75dac-e54c-4d65-af79-ec08b7cf50c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0e548f03c6847e385930f775121d81b'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:32.141107 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1411/2838] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:22:31 2022] DELETE /identity/v3/users/e0e548f03c6847e385930f775121d81b => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:32.435575 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f5721-8c72-41c6-a3e9-88508260eced admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:32.437197 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-120f5721-8c72-41c6-a3e9-88508260eced admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:32.451482 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f5721-8c72-41c6-a3e9-88508260eced admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:32.480768 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:32.522177 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c74fc-10e3-4277-913d-10003428c461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:32.522892 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c74fc-10e3-4277-913d-10003428c461 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:32.523178 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c74fc-10e3-4277-913d-10003428c461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:32.523454 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c74fc-10e3-4277-913d-10003428c461 admin admin] PATH_INFO: `/v3/projects/aa8bbfee5f424d94bc84fd0457d81ddf` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:32.524658 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74c74fc-10e3-4277-913d-10003428c461 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa8bbfee5f424d94bc84fd0457d81ddf)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:32.526973 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74c74fc-10e3-4277-913d-10003428c461 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:32.578269 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-b74c74fc-10e3-4277-913d-10003428c461 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa8bbfee5f424d94bc84fd0457d81ddf', 'request_id': 'req-b74c74fc-10e3-4277-913d-10003428c461'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:32.584917 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-b74c74fc-10e3-4277-913d-10003428c461 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa8bbfee5f424d94bc84fd0457d81ddf', 'request_id': 'req-b74c74fc-10e3-4277-913d-10003428c461'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:32.590779 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-b74c74fc-10e3-4277-913d-10003428c461 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa8bbfee5f424d94bc84fd0457d81ddf', 'request_id': 'req-b74c74fc-10e3-4277-913d-10003428c461'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:32.603195 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-b74c74fc-10e3-4277-913d-10003428c461 admin admin] The token cache is being invalidate because project aa8bbfee5f424d94bc84fd0457d81ddf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:22:32.603482 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-b74c74fc-10e3-4277-913d-10003428c461 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:32.604701 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-b74c74fc-10e3-4277-913d-10003428c461 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:32.606182 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1428/2839] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:22:32 2022] DELETE /identity/v3/projects/aa8bbfee5f424d94bc84fd0457d81ddf => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:32.612494 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71adb4a9-c788-4df8-8367-31144d0ca409 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:32.612972 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71adb4a9-c788-4df8-8367-31144d0ca409 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:32.613115 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71adb4a9-c788-4df8-8367-31144d0ca409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:32.634374 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-71adb4a9-c788-4df8-8367-31144d0ca409 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:32.635743 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-71adb4a9-c788-4df8-8367-31144d0ca409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:32.697198 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1412/2840] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:22:32.706496 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ad53b-dce3-4fab-8d84-e9b4f8b21cae None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:32.706781 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ad53b-dce3-4fab-8d84-e9b4f8b21cae None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:32.707056 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ad53b-dce3-4fab-8d84-e9b4f8b21cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:32.727092 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-539ad53b-dce3-4fab-8d84-e9b4f8b21cae None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:32.728324 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-539ad53b-dce3-4fab-8d84-e9b4f8b21cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:32.737386 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1429/2841] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:22:32.741889 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71adb4a9-c788-4df8-8367-31144d0ca409 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:32.752080 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71adb4a9-c788-4df8-8367-31144d0ca409 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:32.756765 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5fbf7a-cef6-426c-9a63-2734522814ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:32.757366 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5fbf7a-cef6-426c-9a63-2734522814ba admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:32.757644 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5fbf7a-cef6-426c-9a63-2734522814ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:32.757915 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5fbf7a-cef6-426c-9a63-2734522814ba admin admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:32.758852 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5fbf7a-cef6-426c-9a63-2734522814ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:22:32.759155 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5fbf7a-cef6-426c-9a63-2734522814ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:32.761073 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5fbf7a-cef6-426c-9a63-2734522814ba admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:32.772619 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1413/2842] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:22:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:22:32.777622 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539ad53b-dce3-4fab-8d84-e9b4f8b21cae None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:32.786851 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539ad53b-dce3-4fab-8d84-e9b4f8b21cae None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:32.791454 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414f120c-626a-49a5-9149-2467fc8dd028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:32.792863 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414f120c-626a-49a5-9149-2467fc8dd028 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:32.793143 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414f120c-626a-49a5-9149-2467fc8dd028 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:32.793407 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414f120c-626a-49a5-9149-2467fc8dd028 admin admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:32.794045 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414f120c-626a-49a5-9149-2467fc8dd028 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:32.795922 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414f120c-626a-49a5-9149-2467fc8dd028 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:32.828230 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1430/2843] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:22:32 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:22:32.832409 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5fbf7a-cef6-426c-9a63-2734522814ba admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:32.842897 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5fbf7a-cef6-426c-9a63-2734522814ba admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:32.868466 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:32.904595 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef2549b-4fbd-4528-ba44-a1ea6e309619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:32.905202 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2549b-4fbd-4528-ba44-a1ea6e309619 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:32.905505 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2549b-4fbd-4528-ba44-a1ea6e309619 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:32.905759 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2549b-4fbd-4528-ba44-a1ea6e309619 admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:32.906400 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef2549b-4fbd-4528-ba44-a1ea6e309619 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:32.908342 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef2549b-4fbd-4528-ba44-a1ea6e309619 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:32.930616 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1414/2844] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:22:32 2022] POST /identity/v3/users => generated 283 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:22:32.934949 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414f120c-626a-49a5-9149-2467fc8dd028 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:32.945618 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414f120c-626a-49a5-9149-2467fc8dd028 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:32.950287 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad51e5b-de77-416b-8730-4d4311a80e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:32.950883 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad51e5b-de77-416b-8730-4d4311a80e45 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:32.951162 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad51e5b-de77-416b-8730-4d4311a80e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:32.951435 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad51e5b-de77-416b-8730-4d4311a80e45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:32.952068 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad51e5b-de77-416b-8730-4d4311a80e45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:32.953686 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad51e5b-de77-416b-8730-4d4311a80e45 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:32.964181 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1431/2845] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:22:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:22:32.968111 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef2549b-4fbd-4528-ba44-a1ea6e309619 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:32.978196 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef2549b-4fbd-4528-ba44-a1ea6e309619 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:32.984324 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f84f9-2e4d-4192-9969-306c1c8b51e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:32.984965 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f84f9-2e4d-4192-9969-306c1c8b51e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:32.985251 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f84f9-2e4d-4192-9969-306c1c8b51e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:32.985535 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f84f9-2e4d-4192-9969-306c1c8b51e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:32.986190 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5f84f9-2e4d-4192-9969-306c1c8b51e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:32.987892 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5f84f9-2e4d-4192-9969-306c1c8b51e5 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:32.999289 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1415/2846] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:22:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:22:33.003522 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad51e5b-de77-416b-8730-4d4311a80e45 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:33.014162 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad51e5b-de77-416b-8730-4d4311a80e45 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:33.018819 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0e7e31-cf72-4e77-8367-1cb845077b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:33.019467 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e7e31-cf72-4e77-8367-1cb845077b7f admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:33.019746 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e7e31-cf72-4e77-8367-1cb845077b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:33.020020 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e7e31-cf72-4e77-8367-1cb845077b7f admin admin] PATH_INFO: `/v3/projects/07f7f5007bd645f689bebfa0087b2c69/users/24a4f3279dd7457c9e0b404e95f4d559/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:33.036944 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0e7e31-cf72-4e77-8367-1cb845077b7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=07f7f5007bd645f689bebfa0087b2c69, user_id=24a4f3279dd7457c9e0b404e95f4d559, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:33.039524 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0e7e31-cf72-4e77-8367-1cb845077b7f admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:33.053312 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1432/2847] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:22:33 2022] PUT /identity/v3/projects/07f7f5007bd645f689bebfa0087b2c69/users/24a4f3279dd7457c9e0b404e95f4d559/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:33.064297 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b6522-2d6c-4b39-8fb7-b3afed61f3f0 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:33.064585 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b6522-2d6c-4b39-8fb7-b3afed61f3f0 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:33.064857 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b6522-2d6c-4b39-8fb7-b3afed61f3f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:33.096757 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-c38b6522-2d6c-4b39-8fb7-b3afed61f3f0 None None] MFA Rules not processed for user `24a4f3279dd7457c9e0b404e95f4d559`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:33.098176 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-c38b6522-2d6c-4b39-8fb7-b3afed61f3f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:33.164219 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1416/2848] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:22:33.175716 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0e7e31-cf72-4e77-8367-1cb845077b7f admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:33.177600 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-8d0e7e31-cf72-4e77-8367-1cb845077b7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:33.238796 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0e7e31-cf72-4e77-8367-1cb845077b7f admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:33.242436 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b66e60-ffaf-4a72-ae7c-42ce2dc83465 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:33.243033 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b66e60-ffaf-4a72-ae7c-42ce2dc83465 service cinder] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:33.243306 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b66e60-ffaf-4a72-ae7c-42ce2dc83465 service cinder] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:33.243574 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b66e60-ffaf-4a72-ae7c-42ce2dc83465 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:33.248922 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:33.253576 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b66e60-ffaf-4a72-ae7c-42ce2dc83465 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:33.257788 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b66e60-ffaf-4a72-ae7c-42ce2dc83465 service cinder] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:33.270459 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1433/2849] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:22:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:22:39.341625 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38b6522-2d6c-4b39-8fb7-b3afed61f3f0 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:39.351376 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38b6522-2d6c-4b39-8fb7-b3afed61f3f0 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:39.354937 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ee5c03-2b7c-48c8-85ef-038aedf39c3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:39.355541 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ee5c03-2b7c-48c8-85ef-038aedf39c3c service cinder] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:39.355798 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ee5c03-2b7c-48c8-85ef-038aedf39c3c service cinder] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:39.356104 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ee5c03-2b7c-48c8-85ef-038aedf39c3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:39.357836 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-68ee5c03-2b7c-48c8-85ef-038aedf39c3c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:39.418849 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ee5c03-2b7c-48c8-85ef-038aedf39c3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:39.420326 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ee5c03-2b7c-48c8-85ef-038aedf39c3c service cinder] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:39.429477 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1417/2850] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:22:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:22:43.266148 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=3 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:48.192973 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b66e60-ffaf-4a72-ae7c-42ce2dc83465 service cinder] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:48.194443 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-59b66e60-ffaf-4a72-ae7c-42ce2dc83465 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:48.260489 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b66e60-ffaf-4a72-ae7c-42ce2dc83465 service cinder] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:48.265262 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b570d905-809e-46f6-9c08-332658cf86ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:48.265917 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b570d905-809e-46f6-9c08-332658cf86ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:48.266181 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b570d905-809e-46f6-9c08-332658cf86ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:48.266457 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b570d905-809e-46f6-9c08-332658cf86ff admin admin] PATH_INFO: `/v3/users/4ed12c3ff0ea4bb3b74e970d349d3bd4` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:48.267694 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b570d905-809e-46f6-9c08-332658cf86ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ed12c3ff0ea4bb3b74e970d349d3bd4)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:48.269662 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b570d905-809e-46f6-9c08-332658cf86ff admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:48.319280 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-b570d905-809e-46f6-9c08-332658cf86ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ed12c3ff0ea4bb3b74e970d349d3bd4'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:48.331179 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-b570d905-809e-46f6-9c08-332658cf86ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ed12c3ff0ea4bb3b74e970d349d3bd4'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:48.338291 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-b570d905-809e-46f6-9c08-332658cf86ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ed12c3ff0ea4bb3b74e970d349d3bd4'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:48.346221 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-b570d905-809e-46f6-9c08-332658cf86ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ed12c3ff0ea4bb3b74e970d349d3bd4'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:48.346922 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-b570d905-809e-46f6-9c08-332658cf86ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ed12c3ff0ea4bb3b74e970d349d3bd4'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:48.353387 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1434/2851] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:22:48 2022] DELETE /identity/v3/users/4ed12c3ff0ea4bb3b74e970d349d3bd4 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:48.357857 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3781e48-9c29-41b3-9fd9-6fd36a9aaa02 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:48.358362 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3781e48-9c29-41b3-9fd9-6fd36a9aaa02 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:48.358478 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3781e48-9c29-41b3-9fd9-6fd36a9aaa02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:48.378132 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-d3781e48-9c29-41b3-9fd9-6fd36a9aaa02 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:48.379345 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-d3781e48-9c29-41b3-9fd9-6fd36a9aaa02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:48.410093 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:48.441718 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1418/2852] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:22:48.451726 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b570d905-809e-46f6-9c08-332658cf86ff admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:48.453468 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-b570d905-809e-46f6-9c08-332658cf86ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:48.516454 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b570d905-809e-46f6-9c08-332658cf86ff admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:48.520717 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba3305c-5660-4cc6-834b-d7cb50b557c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:48.521394 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba3305c-5660-4cc6-834b-d7cb50b557c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:48.521746 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba3305c-5660-4cc6-834b-d7cb50b557c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:48.522074 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba3305c-5660-4cc6-834b-d7cb50b557c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:48.531804 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba3305c-5660-4cc6-834b-d7cb50b557c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:48.533283 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba3305c-5660-4cc6-834b-d7cb50b557c0 service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:48.542708 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1435/2853] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:22:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:22:48.825833 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3781e48-9c29-41b3-9fd9-6fd36a9aaa02 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:48.827530 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-d3781e48-9c29-41b3-9fd9-6fd36a9aaa02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:48.841918 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3781e48-9c29-41b3-9fd9-6fd36a9aaa02 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:48.847362 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a603c7-e1e4-4f04-bd30-4d3d6330d303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:48.848112 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a603c7-e1e4-4f04-bd30-4d3d6330d303 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:48.848522 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a603c7-e1e4-4f04-bd30-4d3d6330d303 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:48.848899 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a603c7-e1e4-4f04-bd30-4d3d6330d303 admin admin] PATH_INFO: `/v3/projects/921c606e2d594a768ef7419bd69b599f` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:48.850152 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a603c7-e1e4-4f04-bd30-4d3d6330d303 admin admin] RBAC: Authorizing `identity:delete_project(project_id=921c606e2d594a768ef7419bd69b599f)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:48.852529 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a603c7-e1e4-4f04-bd30-4d3d6330d303 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:48.903078 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-c7a603c7-e1e4-4f04-bd30-4d3d6330d303 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '921c606e2d594a768ef7419bd69b599f', 'request_id': 'req-c7a603c7-e1e4-4f04-bd30-4d3d6330d303'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:48.910879 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-c7a603c7-e1e4-4f04-bd30-4d3d6330d303 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '921c606e2d594a768ef7419bd69b599f', 'request_id': 'req-c7a603c7-e1e4-4f04-bd30-4d3d6330d303'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:48.917540 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-c7a603c7-e1e4-4f04-bd30-4d3d6330d303 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '921c606e2d594a768ef7419bd69b599f', 'request_id': 'req-c7a603c7-e1e4-4f04-bd30-4d3d6330d303'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:48.928896 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-c7a603c7-e1e4-4f04-bd30-4d3d6330d303 admin admin] The token cache is being invalidate because project 921c606e2d594a768ef7419bd69b599f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81661) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:22:48.929182 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-c7a603c7-e1e4-4f04-bd30-4d3d6330d303 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:48.929981 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-c7a603c7-e1e4-4f04-bd30-4d3d6330d303 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:48.931598 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1419/2854] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:22:48 2022] DELETE /identity/v3/projects/921c606e2d594a768ef7419bd69b599f => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:48.937945 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e93a950-e022-4618-b1db-c707e067fc3e None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:48.938554 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e93a950-e022-4618-b1db-c707e067fc3e None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:48.938903 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e93a950-e022-4618-b1db-c707e067fc3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:48.959624 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-1e93a950-e022-4618-b1db-c707e067fc3e None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:48.960857 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-1e93a950-e022-4618-b1db-c707e067fc3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:48.980685 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:49.029011 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1436/2855] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:22:49.039203 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12530cf1-8b61-4de6-80cf-e0deaafcc484 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:49.039496 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12530cf1-8b61-4de6-80cf-e0deaafcc484 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:49.039843 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12530cf1-8b61-4de6-80cf-e0deaafcc484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:49.068261 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-12530cf1-8b61-4de6-80cf-e0deaafcc484 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:49.069570 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-12530cf1-8b61-4de6-80cf-e0deaafcc484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:49.079003 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1420/2856] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:22:49.083298 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e93a950-e022-4618-b1db-c707e067fc3e None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:49.092880 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e93a950-e022-4618-b1db-c707e067fc3e None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:49.099157 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0259a84c-5732-46c6-a6d4-4148ef82db4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:49.099841 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0259a84c-5732-46c6-a6d4-4148ef82db4c admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:49.100124 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0259a84c-5732-46c6-a6d4-4148ef82db4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:49.100446 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0259a84c-5732-46c6-a6d4-4148ef82db4c admin admin] PATH_INFO: `/v3/domains` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:49.101123 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0259a84c-5732-46c6-a6d4-4148ef82db4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81660) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:22:49.101400 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0259a84c-5732-46c6-a6d4-4148ef82db4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:49.103000 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0259a84c-5732-46c6-a6d4-4148ef82db4c admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:49.122869 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1437/2857] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:22:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:22:49.126099 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12530cf1-8b61-4de6-80cf-e0deaafcc484 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:49.136863 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12530cf1-8b61-4de6-80cf-e0deaafcc484 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:49.142094 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3716c0c4-f85a-4d86-b2a0-d8814cf8489d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:49.142698 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3716c0c4-f85a-4d86-b2a0-d8814cf8489d admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:49.142982 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3716c0c4-f85a-4d86-b2a0-d8814cf8489d admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:49.143258 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3716c0c4-f85a-4d86-b2a0-d8814cf8489d admin admin] PATH_INFO: `/v3/projects` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:49.144616 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3716c0c4-f85a-4d86-b2a0-d8814cf8489d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:49.146538 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3716c0c4-f85a-4d86-b2a0-d8814cf8489d admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:49.177802 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1421/2858] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:22:49 2022] POST /identity/v3/projects => generated 357 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:22:49.179073 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0259a84c-5732-46c6-a6d4-4148ef82db4c admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:49.189338 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0259a84c-5732-46c6-a6d4-4148ef82db4c admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:49.246673 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cfab94-93a4-4426-b299-795978e35e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:49.247381 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cfab94-93a4-4426-b299-795978e35e33 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:49.247750 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cfab94-93a4-4426-b299-795978e35e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:49.248037 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cfab94-93a4-4426-b299-795978e35e33 admin admin] PATH_INFO: `/v3/users` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:49.248692 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cfab94-93a4-4426-b299-795978e35e33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:49.250948 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cfab94-93a4-4426-b299-795978e35e33 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:49.272771 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1438/2859] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:22:49 2022] POST /identity/v3/users => generated 279 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 01:22:49.276965 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3716c0c4-f85a-4d86-b2a0-d8814cf8489d admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:49.287916 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3716c0c4-f85a-4d86-b2a0-d8814cf8489d admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:49.292526 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228590bb-87bb-4023-8e2d-e3df88180a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:49.293054 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228590bb-87bb-4023-8e2d-e3df88180a55 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:49.293333 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228590bb-87bb-4023-8e2d-e3df88180a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:49.293598 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228590bb-87bb-4023-8e2d-e3df88180a55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:49.294223 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228590bb-87bb-4023-8e2d-e3df88180a55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:49.295818 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228590bb-87bb-4023-8e2d-e3df88180a55 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:49.307312 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1422/2860] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:22:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:22:49.311242 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cfab94-93a4-4426-b299-795978e35e33 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:49.322133 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cfab94-93a4-4426-b299-795978e35e33 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:49.327057 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a175ad8e-0db8-4e87-8a76-9f8d98537292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:49.327804 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a175ad8e-0db8-4e87-8a76-9f8d98537292 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:49.328187 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a175ad8e-0db8-4e87-8a76-9f8d98537292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:49.328365 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a175ad8e-0db8-4e87-8a76-9f8d98537292 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:49.329019 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a175ad8e-0db8-4e87-8a76-9f8d98537292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:49.330617 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a175ad8e-0db8-4e87-8a76-9f8d98537292 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:49.341511 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1439/2861] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:22:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:22:49.346212 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228590bb-87bb-4023-8e2d-e3df88180a55 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:49.356171 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228590bb-87bb-4023-8e2d-e3df88180a55 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:49.365020 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a8d093-2f3e-4cee-98a7-655981073067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:49.365723 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a8d093-2f3e-4cee-98a7-655981073067 admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:49.365989 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a8d093-2f3e-4cee-98a7-655981073067 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:49.366280 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a8d093-2f3e-4cee-98a7-655981073067 admin admin] PATH_INFO: `/v3/projects/fafd225c42e64c36863adfc79edf03d1/users/e7bb6e18952d442b88000d7e64bafe8a/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:49.381885 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a8d093-2f3e-4cee-98a7-655981073067 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fafd225c42e64c36863adfc79edf03d1, user_id=e7bb6e18952d442b88000d7e64bafe8a, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:49.384911 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a8d093-2f3e-4cee-98a7-655981073067 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:49.397979 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1423/2862] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:22:49 2022] PUT /identity/v3/projects/fafd225c42e64c36863adfc79edf03d1/users/e7bb6e18952d442b88000d7e64bafe8a/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:49.409365 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1604a-9c52-4fe4-ad10-00ed84f88909 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:49.409768 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1604a-9c52-4fe4-ad10-00ed84f88909 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:49.410102 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1604a-9c52-4fe4-ad10-00ed84f88909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:49.438867 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-2dd1604a-9c52-4fe4-ad10-00ed84f88909 None None] MFA Rules not processed for user `e7bb6e18952d442b88000d7e64bafe8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:49.440172 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-2dd1604a-9c52-4fe4-ad10-00ed84f88909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:49.440294 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70751a71-8c82-47ba-96d9-74e58f67d1eb None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:49.440529 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70751a71-8c82-47ba-96d9-74e58f67d1eb None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:49.440826 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70751a71-8c82-47ba-96d9-74e58f67d1eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:49.459934 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-70751a71-8c82-47ba-96d9-74e58f67d1eb None None] MFA Rules not processed for user `744fe07ea5644157bdbf11d81adbab89`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:49.461167 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-70751a71-8c82-47ba-96d9-74e58f67d1eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:49.509414 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1440/2863] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:49 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:22:49.510130 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:49.523587 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1424/2864] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:22:49.547988 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd1604a-9c52-4fe4-ad10-00ed84f88909 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:49.549357 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-2dd1604a-9c52-4fe4-ad10-00ed84f88909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:49.571748 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:49.614394 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd1604a-9c52-4fe4-ad10-00ed84f88909 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:49.618126 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f022ed1-de00-45f3-aab6-551c7127527d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:49.618739 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f022ed1-de00-45f3-aab6-551c7127527d service cinder] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:49.619024 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f022ed1-de00-45f3-aab6-551c7127527d service cinder] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:49.619297 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f022ed1-de00-45f3-aab6-551c7127527d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:49.629668 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f022ed1-de00-45f3-aab6-551c7127527d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:49.631132 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f022ed1-de00-45f3-aab6-551c7127527d service cinder] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:49.641531 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1441/2865] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:22:49 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:22:49.657716 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70751a71-8c82-47ba-96d9-74e58f67d1eb None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:49.720433 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70751a71-8c82-47ba-96d9-74e58f67d1eb None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:49.725020 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdde957-f343-4f72-a756-0d683e25ea2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:49.725650 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdde957-f343-4f72-a756-0d683e25ea2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:49.725927 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdde957-f343-4f72-a756-0d683e25ea2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:49.726198 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdde957-f343-4f72-a756-0d683e25ea2a admin admin] PATH_INFO: `/v3/users/e7bb6e18952d442b88000d7e64bafe8a` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:49.727320 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdde957-f343-4f72-a756-0d683e25ea2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7bb6e18952d442b88000d7e64bafe8a)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:49.729304 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdde957-f343-4f72-a756-0d683e25ea2a admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:49.776611 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-3fdde957-f343-4f72-a756-0d683e25ea2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7bb6e18952d442b88000d7e64bafe8a'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:49.790008 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-3fdde957-f343-4f72-a756-0d683e25ea2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7bb6e18952d442b88000d7e64bafe8a'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:49.797319 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-3fdde957-f343-4f72-a756-0d683e25ea2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7bb6e18952d442b88000d7e64bafe8a'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:49.798102 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-3fdde957-f343-4f72-a756-0d683e25ea2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7bb6e18952d442b88000d7e64bafe8a'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:49.808185 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-3fdde957-f343-4f72-a756-0d683e25ea2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7bb6e18952d442b88000d7e64bafe8a'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:49.815056 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1425/2866] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:22:49 2022] DELETE /identity/v3/users/e7bb6e18952d442b88000d7e64bafe8a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:49.819522 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a405fa-d7df-401d-ba75-dc31223c99b2 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:49.819812 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a405fa-d7df-401d-ba75-dc31223c99b2 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:49.820093 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a405fa-d7df-401d-ba75-dc31223c99b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:49.839642 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-f6a405fa-d7df-401d-ba75-dc31223c99b2 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:49.840896 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-f6a405fa-d7df-401d-ba75-dc31223c99b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:49.903531 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1442/2867] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:22:49.913620 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdde957-f343-4f72-a756-0d683e25ea2a admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:49.915046 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-3fdde957-f343-4f72-a756-0d683e25ea2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:49.974599 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdde957-f343-4f72-a756-0d683e25ea2a admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:49.978336 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c3fdc-5d0f-4031-a1c6-665b680c39ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:49.978940 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c3fdc-5d0f-4031-a1c6-665b680c39ba service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:49.979218 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c3fdc-5d0f-4031-a1c6-665b680c39ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:49.979490 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c3fdc-5d0f-4031-a1c6-665b680c39ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:49.988706 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1c3fdc-5d0f-4031-a1c6-665b680c39ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:49.990147 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1c3fdc-5d0f-4031-a1c6-665b680c39ba service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:50.000173 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1426/2868] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:22:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:22:50.290546 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a405fa-d7df-401d-ba75-dc31223c99b2 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:50.291980 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-f6a405fa-d7df-401d-ba75-dc31223c99b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:50.305048 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a405fa-d7df-401d-ba75-dc31223c99b2 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:50.311324 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5f638f-d146-45dd-93cf-cab610ac70d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:50.312199 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f638f-d146-45dd-93cf-cab610ac70d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:50.312566 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f638f-d146-45dd-93cf-cab610ac70d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:50.312944 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f638f-d146-45dd-93cf-cab610ac70d6 admin admin] PATH_INFO: `/v3/projects/fafd225c42e64c36863adfc79edf03d1` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:50.314788 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5f638f-d146-45dd-93cf-cab610ac70d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fafd225c42e64c36863adfc79edf03d1)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:50.316365 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5f638f-d146-45dd-93cf-cab610ac70d6 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:50.376090 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-ab5f638f-d146-45dd-93cf-cab610ac70d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fafd225c42e64c36863adfc79edf03d1', 'request_id': 'req-ab5f638f-d146-45dd-93cf-cab610ac70d6'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:50.390847 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-ab5f638f-d146-45dd-93cf-cab610ac70d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fafd225c42e64c36863adfc79edf03d1', 'request_id': 'req-ab5f638f-d146-45dd-93cf-cab610ac70d6'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:50.397283 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-ab5f638f-d146-45dd-93cf-cab610ac70d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fafd225c42e64c36863adfc79edf03d1', 'request_id': 'req-ab5f638f-d146-45dd-93cf-cab610ac70d6'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:50.411384 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-ab5f638f-d146-45dd-93cf-cab610ac70d6 admin admin] The token cache is being invalidate because project fafd225c42e64c36863adfc79edf03d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:22:50.411668 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-ab5f638f-d146-45dd-93cf-cab610ac70d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:50.412336 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-ab5f638f-d146-45dd-93cf-cab610ac70d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:50.413809 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1443/2869] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:22:50 2022] DELETE /identity/v3/projects/fafd225c42e64c36863adfc79edf03d1 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:50.418935 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f2891-fd62-4034-aa54-5f9ed2bc5b35 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:50.419398 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f2891-fd62-4034-aa54-5f9ed2bc5b35 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:50.419521 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f2891-fd62-4034-aa54-5f9ed2bc5b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:50.438980 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-507f2891-fd62-4034-aa54-5f9ed2bc5b35 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:50.440225 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-507f2891-fd62-4034-aa54-5f9ed2bc5b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:50.453914 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:50.500915 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1427/2870] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:22:50.509157 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d98c5c7-11f5-4be2-8f42-270356cea7a2 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:50.509439 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d98c5c7-11f5-4be2-8f42-270356cea7a2 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:50.509740 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d98c5c7-11f5-4be2-8f42-270356cea7a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:50.530042 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-4d98c5c7-11f5-4be2-8f42-270356cea7a2 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:50.532219 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-4d98c5c7-11f5-4be2-8f42-270356cea7a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:50.539935 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1444/2871] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:22:50.544163 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507f2891-fd62-4034-aa54-5f9ed2bc5b35 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:50.553860 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507f2891-fd62-4034-aa54-5f9ed2bc5b35 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:50.558512 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68334415-f32b-4b75-9e24-5f367cee9cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:50.559041 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68334415-f32b-4b75-9e24-5f367cee9cc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:50.559318 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68334415-f32b-4b75-9e24-5f367cee9cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:50.559588 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68334415-f32b-4b75-9e24-5f367cee9cc9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:50.560252 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68334415-f32b-4b75-9e24-5f367cee9cc9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81661) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 01:22:50.560554 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68334415-f32b-4b75-9e24-5f367cee9cc9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:50.562697 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68334415-f32b-4b75-9e24-5f367cee9cc9 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:50.574794 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1428/2872] 10.222.0.42 () {58 vars in 1153 bytes} [Thu Sep 8 01:22:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 01:22:50.579784 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d98c5c7-11f5-4be2-8f42-270356cea7a2 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:50.589098 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d98c5c7-11f5-4be2-8f42-270356cea7a2 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:50.594455 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1dccdb-026b-4c2f-9d4c-d8d9c12f70c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:50.595047 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1dccdb-026b-4c2f-9d4c-d8d9c12f70c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:50.595325 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1dccdb-026b-4c2f-9d4c-d8d9c12f70c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:50.595731 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1dccdb-026b-4c2f-9d4c-d8d9c12f70c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:50.596483 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1dccdb-026b-4c2f-9d4c-d8d9c12f70c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:50.598419 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1dccdb-026b-4c2f-9d4c-d8d9c12f70c0 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:50.626262 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1445/2873] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:22:50 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:22:50.630526 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68334415-f32b-4b75-9e24-5f367cee9cc9 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:50.641891 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68334415-f32b-4b75-9e24-5f367cee9cc9 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:50.700261 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd00de7-28df-4994-b7ea-ed5497632b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:50.700892 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd00de7-28df-4994-b7ea-ed5497632b65 admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:50.701175 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd00de7-28df-4994-b7ea-ed5497632b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:50.701461 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd00de7-28df-4994-b7ea-ed5497632b65 admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:50.702078 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd00de7-28df-4994-b7ea-ed5497632b65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:50.704594 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd00de7-28df-4994-b7ea-ed5497632b65 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:50.726527 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1429/2874] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:22:50 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:22:50.730663 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1dccdb-026b-4c2f-9d4c-d8d9c12f70c0 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:50.741562 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1dccdb-026b-4c2f-9d4c-d8d9c12f70c0 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:50.746354 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ae03e-1608-4988-9161-ecbea3646118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:50.746954 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ae03e-1608-4988-9161-ecbea3646118 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:50.747233 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ae03e-1608-4988-9161-ecbea3646118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:50.747504 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ae03e-1608-4988-9161-ecbea3646118 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:50.748169 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095ae03e-1608-4988-9161-ecbea3646118 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:50.750435 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095ae03e-1608-4988-9161-ecbea3646118 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:50.760807 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1446/2875] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:22:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 01:22:50.764963 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd00de7-28df-4994-b7ea-ed5497632b65 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:50.775567 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd00de7-28df-4994-b7ea-ed5497632b65 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:50.780277 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4bfaf-8498-4048-b88f-b6b1c45f11f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:50.780899 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4bfaf-8498-4048-b88f-b6b1c45f11f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:50.781183 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4bfaf-8498-4048-b88f-b6b1c45f11f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:50.781567 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4bfaf-8498-4048-b88f-b6b1c45f11f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:50.782206 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf4bfaf-8498-4048-b88f-b6b1c45f11f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:50.783814 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf4bfaf-8498-4048-b88f-b6b1c45f11f7 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:50.794318 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1430/2876] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:22:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:22:50.798277 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ae03e-1608-4988-9161-ecbea3646118 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:50.805016 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:50.809078 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ae03e-1608-4988-9161-ecbea3646118 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:50.816062 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e545a181-f1b5-4318-a100-47135d1bd39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:50.816934 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545a181-f1b5-4318-a100-47135d1bd39e admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:50.817273 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545a181-f1b5-4318-a100-47135d1bd39e admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:50.817726 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545a181-f1b5-4318-a100-47135d1bd39e admin admin] PATH_INFO: `/v3/projects/ec87983f386f4e77a4fece3e45dd56a3/users/fac43bd2278842ccb7281c4f8bbd6716/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:50.832691 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e545a181-f1b5-4318-a100-47135d1bd39e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec87983f386f4e77a4fece3e45dd56a3, user_id=fac43bd2278842ccb7281c4f8bbd6716, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:50.835340 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e545a181-f1b5-4318-a100-47135d1bd39e admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:50.849451 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1447/2877] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:22:50 2022] PUT /identity/v3/projects/ec87983f386f4e77a4fece3e45dd56a3/users/fac43bd2278842ccb7281c4f8bbd6716/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:50.861441 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b989c75-c88e-4c85-a744-37421ff56a8e None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:50.861752 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b989c75-c88e-4c85-a744-37421ff56a8e None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:50.862025 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b989c75-c88e-4c85-a744-37421ff56a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:50.892110 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-9b989c75-c88e-4c85-a744-37421ff56a8e None None] MFA Rules not processed for user `fac43bd2278842ccb7281c4f8bbd6716`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:50.893342 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-9b989c75-c88e-4c85-a744-37421ff56a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:50.956776 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1431/2878] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:22:50.961273 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e545a181-f1b5-4318-a100-47135d1bd39e admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:51.023196 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e545a181-f1b5-4318-a100-47135d1bd39e admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:51.027906 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92d1b4-374c-41bf-9e40-93114415aace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:51.028539 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92d1b4-374c-41bf-9e40-93114415aace admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:51.028823 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92d1b4-374c-41bf-9e40-93114415aace admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:51.029097 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92d1b4-374c-41bf-9e40-93114415aace admin admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:51.029732 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e92d1b4-374c-41bf-9e40-93114415aace admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:51.031638 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e92d1b4-374c-41bf-9e40-93114415aace admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:51.057914 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1448/2879] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:22:50 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 01:22:51.062836 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b989c75-c88e-4c85-a744-37421ff56a8e None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:51.073410 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b989c75-c88e-4c85-a744-37421ff56a8e None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:51.084904 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:51.131379 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b06bcd-2d33-44aa-90e4-e8c316ab3c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:51.131974 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b06bcd-2d33-44aa-90e4-e8c316ab3c2f admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:51.132283 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b06bcd-2d33-44aa-90e4-e8c316ab3c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:51.132554 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b06bcd-2d33-44aa-90e4-e8c316ab3c2f admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:51.133437 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b06bcd-2d33-44aa-90e4-e8c316ab3c2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:51.135357 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b06bcd-2d33-44aa-90e4-e8c316ab3c2f admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:51.155991 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1432/2880] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:22:51 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 01:22:51.160214 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92d1b4-374c-41bf-9e40-93114415aace admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:51.170539 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92d1b4-374c-41bf-9e40-93114415aace admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:51.176293 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25f520e-d2ba-445d-bce9-14fb91bdd315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:51.176916 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f520e-d2ba-445d-bce9-14fb91bdd315 admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:51.177216 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f520e-d2ba-445d-bce9-14fb91bdd315 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:51.177495 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f520e-d2ba-445d-bce9-14fb91bdd315 admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:51.178162 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25f520e-d2ba-445d-bce9-14fb91bdd315 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:51.179753 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25f520e-d2ba-445d-bce9-14fb91bdd315 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:51.190930 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1449/2881] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:22:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:22:51.194743 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b06bcd-2d33-44aa-90e4-e8c316ab3c2f admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:51.204849 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b06bcd-2d33-44aa-90e4-e8c316ab3c2f admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:51.209768 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c8536-d313-4e82-bf36-566de3e16f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:51.210365 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c8536-d313-4e82-bf36-566de3e16f1c admin admin] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:51.210675 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c8536-d313-4e82-bf36-566de3e16f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:51.210945 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c8536-d313-4e82-bf36-566de3e16f1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:51.211581 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872c8536-d313-4e82-bf36-566de3e16f1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:51.213262 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872c8536-d313-4e82-bf36-566de3e16f1c admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:51.223163 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1433/2882] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:22:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:22:51.227120 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25f520e-d2ba-445d-bce9-14fb91bdd315 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:51.237971 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25f520e-d2ba-445d-bce9-14fb91bdd315 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:51.242619 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd0bef2-2a05-4833-8b88-fdc9e5493cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:51.243259 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd0bef2-2a05-4833-8b88-fdc9e5493cc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:51.243537 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd0bef2-2a05-4833-8b88-fdc9e5493cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:51.243809 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd0bef2-2a05-4833-8b88-fdc9e5493cc7 admin admin] PATH_INFO: `/v3/projects/9f039ed4b7674a68a3149e50975a07fd/users/bc9f59be1e664981ae128a2ac88aaa2f/roles/92aa6d6824404d10b4a3be8b4a6bc317` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:51.259211 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd0bef2-2a05-4833-8b88-fdc9e5493cc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f039ed4b7674a68a3149e50975a07fd, user_id=bc9f59be1e664981ae128a2ac88aaa2f, role_id=92aa6d6824404d10b4a3be8b4a6bc317)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:51.261916 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd0bef2-2a05-4833-8b88-fdc9e5493cc7 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:51.275027 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1450/2883] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:22:51 2022] PUT /identity/v3/projects/9f039ed4b7674a68a3149e50975a07fd/users/bc9f59be1e664981ae128a2ac88aaa2f/roles/92aa6d6824404d10b4a3be8b4a6bc317 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:51.285474 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c85db45-8c3e-45a2-be64-d8737092dac5 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:51.285759 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c85db45-8c3e-45a2-be64-d8737092dac5 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:51.286034 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c85db45-8c3e-45a2-be64-d8737092dac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:51.317010 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-9c85db45-8c3e-45a2-be64-d8737092dac5 None None] MFA Rules not processed for user `bc9f59be1e664981ae128a2ac88aaa2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:51.318184 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-9c85db45-8c3e-45a2-be64-d8737092dac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:51.388327 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1434/2884] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:22:51.392978 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd0bef2-2a05-4833-8b88-fdc9e5493cc7 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:51.464958 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd0bef2-2a05-4833-8b88-fdc9e5493cc7 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:51.471188 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a038ac02-bf53-4665-8592-d4af399756e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:51.472067 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038ac02-bf53-4665-8592-d4af399756e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:51.472373 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038ac02-bf53-4665-8592-d4af399756e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:51.472643 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038ac02-bf53-4665-8592-d4af399756e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:51.473273 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a038ac02-bf53-4665-8592-d4af399756e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:51.479195 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a038ac02-bf53-4665-8592-d4af399756e8 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:51.508700 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1451/2885] 10.222.0.42 () {60 vars in 1153 bytes} [Thu Sep 8 01:22:51 2022] POST /identity/v3/projects => generated 383 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:22:51.512887 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c85db45-8c3e-45a2-be64-d8737092dac5 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:51.527669 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c85db45-8c3e-45a2-be64-d8737092dac5 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:51.584833 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07067ca-9750-4085-9205-c4f59336f47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:51.585356 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07067ca-9750-4085-9205-c4f59336f47a admin admin] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:51.585634 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07067ca-9750-4085-9205-c4f59336f47a admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:51.585904 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07067ca-9750-4085-9205-c4f59336f47a admin admin] PATH_INFO: `/v3/users` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:51.586537 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07067ca-9750-4085-9205-c4f59336f47a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:51.588479 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07067ca-9750-4085-9205-c4f59336f47a admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:51.611306 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1435/2886] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Sep 8 01:22:51 2022] POST /identity/v3/users => generated 298 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 01:22:51.615323 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a038ac02-bf53-4665-8592-d4af399756e8 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:51.625816 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a038ac02-bf53-4665-8592-d4af399756e8 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:51.630656 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5597e9e2-3f0c-4cb7-a175-1000284610ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:51.631263 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5597e9e2-3f0c-4cb7-a175-1000284610ff admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:51.631544 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5597e9e2-3f0c-4cb7-a175-1000284610ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:51.631820 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5597e9e2-3f0c-4cb7-a175-1000284610ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:51.632498 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5597e9e2-3f0c-4cb7-a175-1000284610ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:51.634363 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5597e9e2-3f0c-4cb7-a175-1000284610ff admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:51.645991 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1452/2887] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:22:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 01:22:51.650133 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07067ca-9750-4085-9205-c4f59336f47a admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:51.660636 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07067ca-9750-4085-9205-c4f59336f47a admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:51.665251 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13022d08-4c27-4c59-a787-f8ff1c52d474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:51.665897 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13022d08-4c27-4c59-a787-f8ff1c52d474 admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:51.666178 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13022d08-4c27-4c59-a787-f8ff1c52d474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:51.666465 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13022d08-4c27-4c59-a787-f8ff1c52d474 admin admin] PATH_INFO: `/v3/projects/0ea23ea81dbd4b71aebe7103d92db46f/users/22ffbec70bde4038b648248868748108/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:51.678169 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:51.682504 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13022d08-4c27-4c59-a787-f8ff1c52d474 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ea23ea81dbd4b71aebe7103d92db46f, user_id=22ffbec70bde4038b648248868748108, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:51.685666 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13022d08-4c27-4c59-a787-f8ff1c52d474 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:51.698466 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1436/2888] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:22:51 2022] PUT /identity/v3/projects/0ea23ea81dbd4b71aebe7103d92db46f/users/22ffbec70bde4038b648248868748108/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:51.703268 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5597e9e2-3f0c-4cb7-a175-1000284610ff admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:51.710170 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:51.714645 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5597e9e2-3f0c-4cb7-a175-1000284610ff admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:51.775581 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f12f18-52df-4290-ac91-ce89f45bbbdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:51.776213 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f12f18-52df-4290-ac91-ce89f45bbbdf admin admin] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:51.776499 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f12f18-52df-4290-ac91-ce89f45bbbdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:51.776777 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f12f18-52df-4290-ac91-ce89f45bbbdf admin admin] PATH_INFO: `/v3/roles` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:51.777412 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f12f18-52df-4290-ac91-ce89f45bbbdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:51.779053 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f12f18-52df-4290-ac91-ce89f45bbbdf admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:51.789670 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1453/2889] 10.222.0.42 () {58 vars in 1122 bytes} [Thu Sep 8 01:22:51 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 01:22:51.794046 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13022d08-4c27-4c59-a787-f8ff1c52d474 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:51.805229 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13022d08-4c27-4c59-a787-f8ff1c52d474 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:51.809969 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bc6b82-3b9c-4f30-b9f8-a283e8d83e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:51.810638 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc6b82-3b9c-4f30-b9f8-a283e8d83e61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:51.810908 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc6b82-3b9c-4f30-b9f8-a283e8d83e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:51.811193 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc6b82-3b9c-4f30-b9f8-a283e8d83e61 admin admin] PATH_INFO: `/v3/projects/0ea23ea81dbd4b71aebe7103d92db46f/users/22ffbec70bde4038b648248868748108/roles/34de369ec7c54715aa83a9052a26c3f2` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:51.813283 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bc6b82-3b9c-4f30-b9f8-a283e8d83e61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ea23ea81dbd4b71aebe7103d92db46f, user_id=22ffbec70bde4038b648248868748108, role_id=34de369ec7c54715aa83a9052a26c3f2)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:51.815994 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bc6b82-3b9c-4f30-b9f8-a283e8d83e61 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:51.837042 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1437/2890] 10.222.0.42 () {60 vars in 1483 bytes} [Thu Sep 8 01:22:51 2022] PUT /identity/v3/projects/0ea23ea81dbd4b71aebe7103d92db46f/users/22ffbec70bde4038b648248868748108/roles/34de369ec7c54715aa83a9052a26c3f2 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 01:22:51.849047 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd599f-6bad-4d0f-8ed6-b25d6e53edb8 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:51.849338 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd599f-6bad-4d0f-8ed6-b25d6e53edb8 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:51.849617 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd599f-6bad-4d0f-8ed6-b25d6e53edb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:51.878252 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-76dd599f-6bad-4d0f-8ed6-b25d6e53edb8 None None] MFA Rules not processed for user `22ffbec70bde4038b648248868748108`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:51.879490 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-76dd599f-6bad-4d0f-8ed6-b25d6e53edb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:51.949247 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1454/2891] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:51 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:22:51.961544 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bc6b82-3b9c-4f30-b9f8-a283e8d83e61 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:51.963590 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-84bc6b82-3b9c-4f30-b9f8-a283e8d83e61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:52.022312 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bc6b82-3b9c-4f30-b9f8-a283e8d83e61 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:52.026067 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e004bd8d-0d61-48d5-b78e-dcec7edf4069 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:52.026673 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e004bd8d-0d61-48d5-b78e-dcec7edf4069 service cinder] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:52.026977 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e004bd8d-0d61-48d5-b78e-dcec7edf4069 service cinder] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:52.027248 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e004bd8d-0d61-48d5-b78e-dcec7edf4069 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:52.083743 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e004bd8d-0d61-48d5-b78e-dcec7edf4069 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:52.085220 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e004bd8d-0d61-48d5-b78e-dcec7edf4069 service cinder] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:52.094386 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1438/2892] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:22:51 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:22:53.660468 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dd599f-6bad-4d0f-8ed6-b25d6e53edb8 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:53.670829 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dd599f-6bad-4d0f-8ed6-b25d6e53edb8 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:53.675178 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b9dca7-ba21-4227-9921-d0d13c9552fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:53.675787 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b9dca7-ba21-4227-9921-d0d13c9552fb service cinder] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:53.676068 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b9dca7-ba21-4227-9921-d0d13c9552fb service cinder] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:53.676382 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b9dca7-ba21-4227-9921-d0d13c9552fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:53.731727 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b9dca7-ba21-4227-9921-d0d13c9552fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:53.733194 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b9dca7-ba21-4227-9921-d0d13c9552fb service cinder] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:53.742662 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1455/2893] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:22:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:22:54.289556 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e004bd8d-0d61-48d5-b78e-dcec7edf4069 service cinder] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:54.298972 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e004bd8d-0d61-48d5-b78e-dcec7edf4069 service cinder] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:54.304573 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f4efc1-1545-4c3d-8538-402d236ee712 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a307e9150ead4877b970e09f36553968', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:54.305360 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4efc1-1545-4c3d-8538-402d236ee712 service cinder] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:54.305770 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4efc1-1545-4c3d-8538-402d236ee712 service cinder] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:54.306177 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4efc1-1545-4c3d-8538-402d236ee712 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:54.316146 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f4efc1-1545-4c3d-8538-402d236ee712 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:54.317631 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f4efc1-1545-4c3d-8538-402d236ee712 service cinder] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:54.327029 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1439/2894] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:22:54 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:22:58.294703 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b9dca7-ba21-4227-9921-d0d13c9552fb service cinder] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:58.343021 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=100 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:58.382562 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b9dca7-ba21-4227-9921-d0d13c9552fb service cinder] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:58.387890 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb58afa-74c9-4061-895d-a3d9a0cd92e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:58.388559 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb58afa-74c9-4061-895d-a3d9a0cd92e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:58.388840 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb58afa-74c9-4061-895d-a3d9a0cd92e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:58.389120 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb58afa-74c9-4061-895d-a3d9a0cd92e1 admin admin] PATH_INFO: `/v3/users/fac43bd2278842ccb7281c4f8bbd6716` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:58.390883 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb58afa-74c9-4061-895d-a3d9a0cd92e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fac43bd2278842ccb7281c4f8bbd6716)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:58.393414 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb58afa-74c9-4061-895d-a3d9a0cd92e1 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:58.445014 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-2fb58afa-74c9-4061-895d-a3d9a0cd92e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fac43bd2278842ccb7281c4f8bbd6716'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:58.457234 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-2fb58afa-74c9-4061-895d-a3d9a0cd92e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fac43bd2278842ccb7281c4f8bbd6716'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:58.464316 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-2fb58afa-74c9-4061-895d-a3d9a0cd92e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fac43bd2278842ccb7281c4f8bbd6716'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:58.472921 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-2fb58afa-74c9-4061-895d-a3d9a0cd92e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fac43bd2278842ccb7281c4f8bbd6716'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:58.473669 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-2fb58afa-74c9-4061-895d-a3d9a0cd92e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fac43bd2278842ccb7281c4f8bbd6716'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:58.479874 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1456/2895] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:22:58 2022] DELETE /identity/v3/users/fac43bd2278842ccb7281c4f8bbd6716 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:58.484914 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766f12df-d9d0-4397-b4b5-da05d2295a86 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:58.485432 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766f12df-d9d0-4397-b4b5-da05d2295a86 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:58.485592 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766f12df-d9d0-4397-b4b5-da05d2295a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:58.505424 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-766f12df-d9d0-4397-b4b5-da05d2295a86 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:22:58.506633 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-766f12df-d9d0-4397-b4b5-da05d2295a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:58.567823 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1440/2896] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:22:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 01:22:58.578763 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb58afa-74c9-4061-895d-a3d9a0cd92e1 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:58.580302 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-2fb58afa-74c9-4061-895d-a3d9a0cd92e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:58.639515 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb58afa-74c9-4061-895d-a3d9a0cd92e1 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:58.643327 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c58d4-da2d-4509-8c88-5aa5eab26f6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:58.643933 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c58d4-da2d-4509-8c88-5aa5eab26f6c service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:58.644240 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c58d4-da2d-4509-8c88-5aa5eab26f6c service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:58.644512 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c58d4-da2d-4509-8c88-5aa5eab26f6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:58.653719 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17c58d4-da2d-4509-8c88-5aa5eab26f6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:58.655149 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17c58d4-da2d-4509-8c88-5aa5eab26f6c service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:58.665609 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1457/2897] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:22:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:22:59.099262 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766f12df-d9d0-4397-b4b5-da05d2295a86 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:59.100726 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-766f12df-d9d0-4397-b4b5-da05d2295a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:59.113393 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766f12df-d9d0-4397-b4b5-da05d2295a86 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:59.117976 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7702a7-77b1-4696-8d87-c45c366defa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:59.118614 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7702a7-77b1-4696-8d87-c45c366defa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:59.118892 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7702a7-77b1-4696-8d87-c45c366defa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:59.119281 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7702a7-77b1-4696-8d87-c45c366defa1 admin admin] PATH_INFO: `/v3/projects/ec87983f386f4e77a4fece3e45dd56a3` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:59.120330 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7702a7-77b1-4696-8d87-c45c366defa1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec87983f386f4e77a4fece3e45dd56a3)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:59.122265 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7702a7-77b1-4696-8d87-c45c366defa1 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:59.137748 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:59.194758 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-be7702a7-77b1-4696-8d87-c45c366defa1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec87983f386f4e77a4fece3e45dd56a3', 'request_id': 'req-be7702a7-77b1-4696-8d87-c45c366defa1'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:59.204078 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-be7702a7-77b1-4696-8d87-c45c366defa1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec87983f386f4e77a4fece3e45dd56a3', 'request_id': 'req-be7702a7-77b1-4696-8d87-c45c366defa1'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:59.212539 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-be7702a7-77b1-4696-8d87-c45c366defa1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec87983f386f4e77a4fece3e45dd56a3', 'request_id': 'req-be7702a7-77b1-4696-8d87-c45c366defa1'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:59.224677 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-be7702a7-77b1-4696-8d87-c45c366defa1 admin admin] The token cache is being invalidate because project ec87983f386f4e77a4fece3e45dd56a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81661) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:22:59.225231 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-be7702a7-77b1-4696-8d87-c45c366defa1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:59.225932 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-be7702a7-77b1-4696-8d87-c45c366defa1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:59.227415 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1441/2898] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:22:59 2022] DELETE /identity/v3/projects/ec87983f386f4e77a4fece3e45dd56a3 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:59.231559 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c58d4-da2d-4509-8c88-5aa5eab26f6c service neutron] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:59.241674 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c58d4-da2d-4509-8c88-5aa5eab26f6c service neutron] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:59.292694 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:22:59.315667 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de36a9a7-636a-4c0c-a95b-50a636c7fa9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:59.316414 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de36a9a7-636a-4c0c-a95b-50a636c7fa9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:59.316606 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de36a9a7-636a-4c0c-a95b-50a636c7fa9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:59.316879 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de36a9a7-636a-4c0c-a95b-50a636c7fa9a admin admin] PATH_INFO: `/v3/users/bc9f59be1e664981ae128a2ac88aaa2f` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:59.317988 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de36a9a7-636a-4c0c-a95b-50a636c7fa9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc9f59be1e664981ae128a2ac88aaa2f)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:59.319905 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de36a9a7-636a-4c0c-a95b-50a636c7fa9a admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:59.388418 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-de36a9a7-636a-4c0c-a95b-50a636c7fa9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc9f59be1e664981ae128a2ac88aaa2f'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:59.401925 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-de36a9a7-636a-4c0c-a95b-50a636c7fa9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc9f59be1e664981ae128a2ac88aaa2f'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:59.410173 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-de36a9a7-636a-4c0c-a95b-50a636c7fa9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc9f59be1e664981ae128a2ac88aaa2f'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:59.424610 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-de36a9a7-636a-4c0c-a95b-50a636c7fa9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc9f59be1e664981ae128a2ac88aaa2f'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:59.425320 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-de36a9a7-636a-4c0c-a95b-50a636c7fa9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc9f59be1e664981ae128a2ac88aaa2f'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:59.431922 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1458/2899] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:22:59 2022] DELETE /identity/v3/users/bc9f59be1e664981ae128a2ac88aaa2f => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:22:59.859424 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7702a7-77b1-4696-8d87-c45c366defa1 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:22:59.860890 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-be7702a7-77b1-4696-8d87-c45c366defa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:22:59.874646 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7702a7-77b1-4696-8d87-c45c366defa1 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:22:59.941009 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c9eeaa-9278-4c55-9953-9f48b2d6bf63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:22:59.941635 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9eeaa-9278-4c55-9953-9f48b2d6bf63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:22:59.941912 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9eeaa-9278-4c55-9953-9f48b2d6bf63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:22:59.942177 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9eeaa-9278-4c55-9953-9f48b2d6bf63 admin admin] PATH_INFO: `/v3/projects/9f039ed4b7674a68a3149e50975a07fd` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:22:59.943280 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c9eeaa-9278-4c55-9953-9f48b2d6bf63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f039ed4b7674a68a3149e50975a07fd)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:22:59.945261 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c9eeaa-9278-4c55-9953-9f48b2d6bf63 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:22:59.990988 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-01c9eeaa-9278-4c55-9953-9f48b2d6bf63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f039ed4b7674a68a3149e50975a07fd', 'request_id': 'req-01c9eeaa-9278-4c55-9953-9f48b2d6bf63'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:22:59.998416 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-01c9eeaa-9278-4c55-9953-9f48b2d6bf63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f039ed4b7674a68a3149e50975a07fd', 'request_id': 'req-01c9eeaa-9278-4c55-9953-9f48b2d6bf63'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:23:00.005633 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-01c9eeaa-9278-4c55-9953-9f48b2d6bf63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f039ed4b7674a68a3149e50975a07fd', 'request_id': 'req-01c9eeaa-9278-4c55-9953-9f48b2d6bf63'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:23:00.016158 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-01c9eeaa-9278-4c55-9953-9f48b2d6bf63 admin admin] The token cache is being invalidate because project 9f039ed4b7674a68a3149e50975a07fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81661) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:23:00.016474 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-01c9eeaa-9278-4c55-9953-9f48b2d6bf63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:23:00.017119 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-01c9eeaa-9278-4c55-9953-9f48b2d6bf63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:23:00.018594 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1442/2900] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:22:59 2022] DELETE /identity/v3/projects/9f039ed4b7674a68a3149e50975a07fd => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:23:00.022758 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de36a9a7-636a-4c0c-a95b-50a636c7fa9a admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:23:00.024093 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-de36a9a7-636a-4c0c-a95b-50a636c7fa9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:23:00.036742 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de36a9a7-636a-4c0c-a95b-50a636c7fa9a admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:23:00.095554 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3199a7-afa4-493f-8261-3cf33b020026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:23:00.096208 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3199a7-afa4-493f-8261-3cf33b020026 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:23:00.096488 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3199a7-afa4-493f-8261-3cf33b020026 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:23:00.096761 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3199a7-afa4-493f-8261-3cf33b020026 admin admin] PATH_INFO: `/v3/users/22ffbec70bde4038b648248868748108` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:23:00.097865 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3199a7-afa4-493f-8261-3cf33b020026 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22ffbec70bde4038b648248868748108)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:23:00.099848 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3199a7-afa4-493f-8261-3cf33b020026 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:23:00.146903 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-5d3199a7-afa4-493f-8261-3cf33b020026 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22ffbec70bde4038b648248868748108'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:23:00.159593 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-5d3199a7-afa4-493f-8261-3cf33b020026 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22ffbec70bde4038b648248868748108'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:23:00.161831 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:23:00.168813 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-5d3199a7-afa4-493f-8261-3cf33b020026 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22ffbec70bde4038b648248868748108'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:23:00.178619 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-5d3199a7-afa4-493f-8261-3cf33b020026 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22ffbec70bde4038b648248868748108'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:23:00.179324 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-5d3199a7-afa4-493f-8261-3cf33b020026 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22ffbec70bde4038b648248868748108'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:23:00.185576 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1459/2901] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:23:00 2022] DELETE /identity/v3/users/22ffbec70bde4038b648248868748108 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:23:00.490348 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c9eeaa-9278-4c55-9953-9f48b2d6bf63 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:23:00.491859 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-01c9eeaa-9278-4c55-9953-9f48b2d6bf63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:23:00.506962 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c9eeaa-9278-4c55-9953-9f48b2d6bf63 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:23:00.545058 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:23:00.576804 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5134e3e7-3a83-4788-84b8-d383286af0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:23:00.578193 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134e3e7-3a83-4788-84b8-d383286af0f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:23:00.578718 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134e3e7-3a83-4788-84b8-d383286af0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:23:00.579452 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134e3e7-3a83-4788-84b8-d383286af0f3 admin admin] PATH_INFO: `/v3/projects/0ea23ea81dbd4b71aebe7103d92db46f` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:23:00.580620 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5134e3e7-3a83-4788-84b8-d383286af0f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ea23ea81dbd4b71aebe7103d92db46f)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:23:00.582570 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5134e3e7-3a83-4788-84b8-d383286af0f3 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:23:00.633506 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-5134e3e7-3a83-4788-84b8-d383286af0f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ea23ea81dbd4b71aebe7103d92db46f', 'request_id': 'req-5134e3e7-3a83-4788-84b8-d383286af0f3'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:23:00.641800 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-5134e3e7-3a83-4788-84b8-d383286af0f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ea23ea81dbd4b71aebe7103d92db46f', 'request_id': 'req-5134e3e7-3a83-4788-84b8-d383286af0f3'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:23:00.648880 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-5134e3e7-3a83-4788-84b8-d383286af0f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ea23ea81dbd4b71aebe7103d92db46f', 'request_id': 'req-5134e3e7-3a83-4788-84b8-d383286af0f3'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:23:00.660084 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-5134e3e7-3a83-4788-84b8-d383286af0f3 admin admin] The token cache is being invalidate because project 0ea23ea81dbd4b71aebe7103d92db46f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81661) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:23:00.660408 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-5134e3e7-3a83-4788-84b8-d383286af0f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:23:00.661052 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-5134e3e7-3a83-4788-84b8-d383286af0f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:23:00.662526 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1443/2902] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:23:00 2022] DELETE /identity/v3/projects/0ea23ea81dbd4b71aebe7103d92db46f => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:23:10.181865 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=5,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:23:10.657403 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=37,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:23:12.280007 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50f1663-b509-4a28-94b0-46ec25357b04 None None] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:23:12.280744 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50f1663-b509-4a28-94b0-46ec25357b04 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:23:12.280744 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50f1663-b509-4a28-94b0-46ec25357b04 None None] PATH_INFO: `/` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:23:12.283693 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1460/2903] 10.222.0.42 () {54 vars in 921 bytes} [Thu Sep 8 01:23:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:23:12.291456 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ac9bc-ed01-4fa5-b392-c1b80e8b9aba None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:23:12.292396 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ac9bc-ed01-4fa5-b392-c1b80e8b9aba None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:23:12.292853 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ac9bc-ed01-4fa5-b392-c1b80e8b9aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:23:12.332251 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-d49ac9bc-ed01-4fa5-b392-c1b80e8b9aba None None] MFA Rules not processed for user `5dbbcc3dbcc745458e97b31c6781ce70`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:23:12.333649 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-d49ac9bc-ed01-4fa5-b392-c1b80e8b9aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:23:12.414728 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1444/2904] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Sep 8 01:23:12 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:23:12.427133 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50f1663-b509-4a28-94b0-46ec25357b04 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:23:12.429019 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-b50f1663-b509-4a28-94b0-46ec25357b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:23:12.443605 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50f1663-b509-4a28-94b0-46ec25357b04 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:23:12.447296 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32491aea-f20c-4c17-97e8-a97a7907b0fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:23:12.447974 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32491aea-f20c-4c17-97e8-a97a7907b0fa service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:23:12.448377 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32491aea-f20c-4c17-97e8-a97a7907b0fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:23:12.448662 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32491aea-f20c-4c17-97e8-a97a7907b0fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:23:12.456617 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32491aea-f20c-4c17-97e8-a97a7907b0fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:23:12.458051 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32491aea-f20c-4c17-97e8-a97a7907b0fa service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:23:12.466174 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1461/2905] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:23:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 01:23:12.559052 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b05de07-633c-4ddc-a004-dac10f2cbe45 None None] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:23:12.559482 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b05de07-633c-4ddc-a004-dac10f2cbe45 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:23:12.559805 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b05de07-633c-4ddc-a004-dac10f2cbe45 None None] PATH_INFO: `/` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:23:12.561255 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1445/2906] 10.222.0.42 () {54 vars in 921 bytes} [Thu Sep 8 01:23:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 01:23:12.567518 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80860341-4dda-48c2-8aa2-007fbfddbc67 None None] REQUEST_METHOD: `POST` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:23:12.567800 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80860341-4dda-48c2-8aa2-007fbfddbc67 None None] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:23:12.568073 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80860341-4dda-48c2-8aa2-007fbfddbc67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:23:12.590174 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.auth.core [None req-80860341-4dda-48c2-8aa2-007fbfddbc67 None None] MFA Rules not processed for user `ae643cfdcb164aa9b474747cd21aa116`. Rule list: `[]` (Enabled: `True`). {{(pid=81660) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:23:12.591382 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-80860341-4dda-48c2-8aa2-007fbfddbc67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:23:12.662315 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1462/2907] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Sep 8 01:23:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:23:12.674514 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b05de07-633c-4ddc-a004-dac10f2cbe45 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:23:12.676563 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-5b05de07-633c-4ddc-a004-dac10f2cbe45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:23:12.695291 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b05de07-633c-4ddc-a004-dac10f2cbe45 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:23:12.700321 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d46ed4-d71a-4df6-a1b5-0da2a8442787 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:23:12.701030 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d46ed4-d71a-4df6-a1b5-0da2a8442787 service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:23:12.701402 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d46ed4-d71a-4df6-a1b5-0da2a8442787 service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:23:12.701766 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d46ed4-d71a-4df6-a1b5-0da2a8442787 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:23:12.713194 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d46ed4-d71a-4df6-a1b5-0da2a8442787 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:23:12.715217 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d46ed4-d71a-4df6-a1b5-0da2a8442787 service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:23:12.728889 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1446/2908] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:23:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:23:14.982619 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80860341-4dda-48c2-8aa2-007fbfddbc67 None None] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:23:14.985180 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-80860341-4dda-48c2-8aa2-007fbfddbc67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:23:15.084645 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80860341-4dda-48c2-8aa2-007fbfddbc67 None None] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:23:15.090822 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1b4ea7-9ad7-49f0-8d33-0eb75fa85c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:23:15.091560 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b4ea7-9ad7-49f0-8d33-0eb75fa85c73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:23:15.091905 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b4ea7-9ad7-49f0-8d33-0eb75fa85c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:23:15.092330 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b4ea7-9ad7-49f0-8d33-0eb75fa85c73 admin admin] PATH_INFO: `/v3/users/24a4f3279dd7457c9e0b404e95f4d559` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:23:15.093874 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1b4ea7-9ad7-49f0-8d33-0eb75fa85c73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24a4f3279dd7457c9e0b404e95f4d559)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:23:15.096909 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1b4ea7-9ad7-49f0-8d33-0eb75fa85c73 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:23:15.174450 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-0a1b4ea7-9ad7-49f0-8d33-0eb75fa85c73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24a4f3279dd7457c9e0b404e95f4d559'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:23:15.198481 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-0a1b4ea7-9ad7-49f0-8d33-0eb75fa85c73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24a4f3279dd7457c9e0b404e95f4d559'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:23:15.208807 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-0a1b4ea7-9ad7-49f0-8d33-0eb75fa85c73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24a4f3279dd7457c9e0b404e95f4d559'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:23:15.218376 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-0a1b4ea7-9ad7-49f0-8d33-0eb75fa85c73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24a4f3279dd7457c9e0b404e95f4d559'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:23:15.219748 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-0a1b4ea7-9ad7-49f0-8d33-0eb75fa85c73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24a4f3279dd7457c9e0b404e95f4d559'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:23:15.221455 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:23:15.227870 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1463/2909] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:23:14 2022] DELETE /identity/v3/users/24a4f3279dd7457c9e0b404e95f4d559 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:23:15.232878 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd37ef9-b6ec-4bed-b9b1-0bac03719fd4 None None] REQUEST_METHOD: `POST` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:23:15.233397 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd37ef9-b6ec-4bed-b9b1-0bac03719fd4 None None] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:23:15.233397 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd37ef9-b6ec-4bed-b9b1-0bac03719fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:23:15.261205 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.auth.core [None req-dcd37ef9-b6ec-4bed-b9b1-0bac03719fd4 None None] MFA Rules not processed for user `51f97eb5e7894117a81b5bd04a11488e`. Rule list: `[]` (Enabled: `True`). {{(pid=81661) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 01:23:15.262435 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-dcd37ef9-b6ec-4bed-b9b1-0bac03719fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:23:15.340193 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1447/2910] 10.222.0.42 () {58 vars in 958 bytes} [Thu Sep 8 01:23:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 01:23:15.350279 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1b4ea7-9ad7-49f0-8d33-0eb75fa85c73 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:23:15.352015 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-0a1b4ea7-9ad7-49f0-8d33-0eb75fa85c73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:23:15.410875 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1b4ea7-9ad7-49f0-8d33-0eb75fa85c73 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:23:15.414707 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c2e722-5102-4a3e-8bfd-48335aef9a0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:23:15.415319 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2e722-5102-4a3e-8bfd-48335aef9a0d service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:23:15.415602 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2e722-5102-4a3e-8bfd-48335aef9a0d service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:23:15.415884 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2e722-5102-4a3e-8bfd-48335aef9a0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:23:15.426142 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c2e722-5102-4a3e-8bfd-48335aef9a0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:23:15.427645 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c2e722-5102-4a3e-8bfd-48335aef9a0d service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:23:15.439021 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1464/2911] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:23:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:23:15.721185 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd37ef9-b6ec-4bed-b9b1-0bac03719fd4 None None] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:23:15.722653 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-dcd37ef9-b6ec-4bed-b9b1-0bac03719fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:23:15.735326 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd37ef9-b6ec-4bed-b9b1-0bac03719fd4 None None] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:23:15.739995 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de638e7b-b00b-4475-b1d3-53eb3ebef329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:23:15.740583 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de638e7b-b00b-4475-b1d3-53eb3ebef329 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:23:15.740850 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de638e7b-b00b-4475-b1d3-53eb3ebef329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:23:15.741121 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de638e7b-b00b-4475-b1d3-53eb3ebef329 admin admin] PATH_INFO: `/v3/projects/07f7f5007bd645f689bebfa0087b2c69` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:23:15.742314 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de638e7b-b00b-4475-b1d3-53eb3ebef329 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07f7f5007bd645f689bebfa0087b2c69)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:23:15.744242 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de638e7b-b00b-4475-b1d3-53eb3ebef329 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:23:15.790698 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-de638e7b-b00b-4475-b1d3-53eb3ebef329 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07f7f5007bd645f689bebfa0087b2c69', 'request_id': 'req-de638e7b-b00b-4475-b1d3-53eb3ebef329'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:23:15.798451 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-de638e7b-b00b-4475-b1d3-53eb3ebef329 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07f7f5007bd645f689bebfa0087b2c69', 'request_id': 'req-de638e7b-b00b-4475-b1d3-53eb3ebef329'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:23:15.804858 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-de638e7b-b00b-4475-b1d3-53eb3ebef329 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07f7f5007bd645f689bebfa0087b2c69', 'request_id': 'req-de638e7b-b00b-4475-b1d3-53eb3ebef329'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:23:15.816360 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-de638e7b-b00b-4475-b1d3-53eb3ebef329 admin admin] The token cache is being invalidate because project 07f7f5007bd645f689bebfa0087b2c69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81661) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:23:15.816648 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-de638e7b-b00b-4475-b1d3-53eb3ebef329 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:23:15.817283 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-de638e7b-b00b-4475-b1d3-53eb3ebef329 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:23:15.818811 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1448/2912] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:23:15 2022] DELETE /identity/v3/projects/07f7f5007bd645f689bebfa0087b2c69 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:23:25.439267 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=27 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:23:25.812354 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:24:33.550587 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c2e722-5102-4a3e-8bfd-48335aef9a0d service neutron] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:24:33.552020 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-98c2e722-5102-4a3e-8bfd-48335aef9a0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:24:33.563822 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c2e722-5102-4a3e-8bfd-48335aef9a0d service neutron] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:24:33.612640 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cddf10c-d4aa-45f2-b210-185c9d0f046f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:24:33.613239 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cddf10c-d4aa-45f2-b210-185c9d0f046f service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:24:33.613546 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cddf10c-d4aa-45f2-b210-185c9d0f046f service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:24:33.613820 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cddf10c-d4aa-45f2-b210-185c9d0f046f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:24:33.615536 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-4cddf10c-d4aa-45f2-b210-185c9d0f046f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:24:33.625091 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cddf10c-d4aa-45f2-b210-185c9d0f046f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:24:33.627082 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cddf10c-d4aa-45f2-b210-185c9d0f046f service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:24:33.635546 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1465/2913] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:24:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:24:43.631500 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=26 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:24:49.730554 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de638e7b-b00b-4475-b1d3-53eb3ebef329 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:24:49.732102 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-de638e7b-b00b-4475-b1d3-53eb3ebef329 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:24:49.823315 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de638e7b-b00b-4475-b1d3-53eb3ebef329 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:24:49.829773 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfde562-3c9e-4fb9-83a5-0a3053afc68a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae643cfdcb164aa9b474747cd21aa116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:24:49.830141 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfde562-3c9e-4fb9-83a5-0a3053afc68a service nova] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:24:49.830467 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfde562-3c9e-4fb9-83a5-0a3053afc68a service nova] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:24:49.830687 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfde562-3c9e-4fb9-83a5-0a3053afc68a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:24:49.833650 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-1dfde562-3c9e-4fb9-83a5-0a3053afc68a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:24:49.848277 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfde562-3c9e-4fb9-83a5-0a3053afc68a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:24:49.850557 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfde562-3c9e-4fb9-83a5-0a3053afc68a service nova] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:24:49.862950 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1449/2914] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:24:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:24:49.925991 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cddf10c-d4aa-45f2-b210-185c9d0f046f service neutron] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:24:49.927536 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-4cddf10c-d4aa-45f2-b210-185c9d0f046f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:24:49.938213 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cddf10c-d4aa-45f2-b210-185c9d0f046f service neutron] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:24:49.941991 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc8c688-bfd2-4f53-a32e-82dafc7be96e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:24:49.943314 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc8c688-bfd2-4f53-a32e-82dafc7be96e service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:24:49.943594 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc8c688-bfd2-4f53-a32e-82dafc7be96e service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:24:49.943863 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc8c688-bfd2-4f53-a32e-82dafc7be96e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:24:49.945705 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-3dc8c688-bfd2-4f53-a32e-82dafc7be96e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:24:49.960807 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc8c688-bfd2-4f53-a32e-82dafc7be96e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:24:49.962253 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc8c688-bfd2-4f53-a32e-82dafc7be96e service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:24:49.974998 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1466/2915] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:24:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 01:24:49.981950 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfde562-3c9e-4fb9-83a5-0a3053afc68a service nova] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:24:49.991847 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfde562-3c9e-4fb9-83a5-0a3053afc68a service nova] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:24:49.995848 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285304f-fd25-4437-b426-e1dd11bc9506 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:24:49.996502 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285304f-fd25-4437-b426-e1dd11bc9506 service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:24:49.996785 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285304f-fd25-4437-b426-e1dd11bc9506 service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:24:49.997128 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285304f-fd25-4437-b426-e1dd11bc9506 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:24:49.999090 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-c285304f-fd25-4437-b426-e1dd11bc9506 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:24:50.061985 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c285304f-fd25-4437-b426-e1dd11bc9506 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:24:50.063422 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c285304f-fd25-4437-b426-e1dd11bc9506 service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:24:50.073059 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1450/2916] 10.222.0.42 () {60 vars in 1396 bytes} [Thu Sep 8 01:24:49 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3248 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:24:51.305944 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc8c688-bfd2-4f53-a32e-82dafc7be96e service neutron] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:24:51.308452 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-3dc8c688-bfd2-4f53-a32e-82dafc7be96e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:24:51.320605 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc8c688-bfd2-4f53-a32e-82dafc7be96e service neutron] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:24:51.324701 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199acfd5-9df7-4fe6-95c0-e86b85d394b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:24:51.325623 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199acfd5-9df7-4fe6-95c0-e86b85d394b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:24:51.326066 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199acfd5-9df7-4fe6-95c0-e86b85d394b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:24:51.326509 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199acfd5-9df7-4fe6-95c0-e86b85d394b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:24:51.328752 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-199acfd5-9df7-4fe6-95c0-e86b85d394b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:24:51.344922 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199acfd5-9df7-4fe6-95c0-e86b85d394b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:24:51.346756 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199acfd5-9df7-4fe6-95c0-e86b85d394b8 service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:24:51.361365 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1467/2917] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:24:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:24:56.266436 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285304f-fd25-4437-b426-e1dd11bc9506 service neutron] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:24:56.276945 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285304f-fd25-4437-b426-e1dd11bc9506 service neutron] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:24:56.281248 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4e561f-3ba2-469f-91f8-a2dd34fad105 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:24:56.281911 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e561f-3ba2-469f-91f8-a2dd34fad105 service neutron] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:24:56.282249 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e561f-3ba2-469f-91f8-a2dd34fad105 service neutron] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:24:56.282553 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e561f-3ba2-469f-91f8-a2dd34fad105 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:24:56.285826 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-9c4e561f-3ba2-469f-91f8-a2dd34fad105 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:24:56.355116 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4e561f-3ba2-469f-91f8-a2dd34fad105 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:24:56.356700 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4e561f-3ba2-469f-91f8-a2dd34fad105 service neutron] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:24:56.366748 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1451/2918] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:24:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:25:01.353906 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=12 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:25:03.941795 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199acfd5-9df7-4fe6-95c0-e86b85d394b8 service neutron] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:25:03.943326 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-199acfd5-9df7-4fe6-95c0-e86b85d394b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:25:03.959004 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199acfd5-9df7-4fe6-95c0-e86b85d394b8 service neutron] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:25:03.963846 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a784c08-5762-457e-8b53-f4390b18d1e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:25:03.964513 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a784c08-5762-457e-8b53-f4390b18d1e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:25:03.964795 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a784c08-5762-457e-8b53-f4390b18d1e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:25:03.965068 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a784c08-5762-457e-8b53-f4390b18d1e1 admin admin] PATH_INFO: `/v3/users/00917dba4a9f4d56be3ec74f90a65154` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:25:03.966199 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a784c08-5762-457e-8b53-f4390b18d1e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00917dba4a9f4d56be3ec74f90a65154)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:25:03.968635 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a784c08-5762-457e-8b53-f4390b18d1e1 admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:25:04.019189 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-7a784c08-5762-457e-8b53-f4390b18d1e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00917dba4a9f4d56be3ec74f90a65154'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:25:04.033557 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-7a784c08-5762-457e-8b53-f4390b18d1e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00917dba4a9f4d56be3ec74f90a65154'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:25:04.041467 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-7a784c08-5762-457e-8b53-f4390b18d1e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00917dba4a9f4d56be3ec74f90a65154'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:25:04.049011 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-7a784c08-5762-457e-8b53-f4390b18d1e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00917dba4a9f4d56be3ec74f90a65154'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:25:04.049838 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-7a784c08-5762-457e-8b53-f4390b18d1e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00917dba4a9f4d56be3ec74f90a65154'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:25:04.056344 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1468/2919] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:25:03 2022] DELETE /identity/v3/users/00917dba4a9f4d56be3ec74f90a65154 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:25:04.267969 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4e561f-3ba2-469f-91f8-a2dd34fad105 service neutron] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:25:04.280509 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4e561f-3ba2-469f-91f8-a2dd34fad105 service neutron] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:25:04.300359 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=100 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:25:04.358844 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61242554-a14a-4177-9212-1515cbee6b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:25:04.359477 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61242554-a14a-4177-9212-1515cbee6b3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:25:04.359751 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61242554-a14a-4177-9212-1515cbee6b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:25:04.360043 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61242554-a14a-4177-9212-1515cbee6b3c admin admin] PATH_INFO: `/v3/projects/e32dd1d6dcf54fbb83667953c4cbea29` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:25:04.361196 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61242554-a14a-4177-9212-1515cbee6b3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e32dd1d6dcf54fbb83667953c4cbea29)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:25:04.363176 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61242554-a14a-4177-9212-1515cbee6b3c admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:25:04.412940 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-61242554-a14a-4177-9212-1515cbee6b3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e32dd1d6dcf54fbb83667953c4cbea29', 'request_id': 'req-61242554-a14a-4177-9212-1515cbee6b3c'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:25:04.420545 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-61242554-a14a-4177-9212-1515cbee6b3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e32dd1d6dcf54fbb83667953c4cbea29', 'request_id': 'req-61242554-a14a-4177-9212-1515cbee6b3c'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:25:04.427514 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-61242554-a14a-4177-9212-1515cbee6b3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e32dd1d6dcf54fbb83667953c4cbea29', 'request_id': 'req-61242554-a14a-4177-9212-1515cbee6b3c'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:25:04.438123 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-61242554-a14a-4177-9212-1515cbee6b3c admin admin] The token cache is being invalidate because project e32dd1d6dcf54fbb83667953c4cbea29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81661) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:25:04.438411 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-61242554-a14a-4177-9212-1515cbee6b3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:25:04.439158 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-61242554-a14a-4177-9212-1515cbee6b3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:25:04.440685 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1452/2920] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:25:04 2022] DELETE /identity/v3/projects/e32dd1d6dcf54fbb83667953c4cbea29 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:25:04.445860 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a784c08-5762-457e-8b53-f4390b18d1e1 admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:25:04.447332 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-7a784c08-5762-457e-8b53-f4390b18d1e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:25:04.461868 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a784c08-5762-457e-8b53-f4390b18d1e1 admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:25:04.517940 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ac21a-84db-4e10-9f8d-8750d436a1ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:25:04.518568 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ac21a-84db-4e10-9f8d-8750d436a1ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:25:04.518848 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ac21a-84db-4e10-9f8d-8750d436a1ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:25:04.519123 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ac21a-84db-4e10-9f8d-8750d436a1ae admin admin] PATH_INFO: `/v3/users/fca710369bd546599a21503e3fb505dc` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:25:04.521084 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58ac21a-84db-4e10-9f8d-8750d436a1ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=fca710369bd546599a21503e3fb505dc)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:25:04.523002 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58ac21a-84db-4e10-9f8d-8750d436a1ae admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:25:04.568434 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-b58ac21a-84db-4e10-9f8d-8750d436a1ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fca710369bd546599a21503e3fb505dc'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:25:04.580114 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-b58ac21a-84db-4e10-9f8d-8750d436a1ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fca710369bd546599a21503e3fb505dc'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:25:04.587900 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-b58ac21a-84db-4e10-9f8d-8750d436a1ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fca710369bd546599a21503e3fb505dc'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:25:04.589561 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:25:04.596197 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-b58ac21a-84db-4e10-9f8d-8750d436a1ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fca710369bd546599a21503e3fb505dc'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:25:04.597073 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-b58ac21a-84db-4e10-9f8d-8750d436a1ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fca710369bd546599a21503e3fb505dc'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:25:04.604340 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1469/2921] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:25:04 2022] DELETE /identity/v3/users/fca710369bd546599a21503e3fb505dc => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:25:04.788413 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61242554-a14a-4177-9212-1515cbee6b3c admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:25:04.789041 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-61242554-a14a-4177-9212-1515cbee6b3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:25:04.811239 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61242554-a14a-4177-9212-1515cbee6b3c admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:25:04.868711 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dccf7f-d57a-4237-ac34-2a48fc6b9025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:25:04.869332 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dccf7f-d57a-4237-ac34-2a48fc6b9025 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:25:04.869609 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dccf7f-d57a-4237-ac34-2a48fc6b9025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:25:04.869874 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dccf7f-d57a-4237-ac34-2a48fc6b9025 admin admin] PATH_INFO: `/v3/projects/4037ed70420a475cb2a0a50d75d15daf` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:25:04.871018 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dccf7f-d57a-4237-ac34-2a48fc6b9025 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4037ed70420a475cb2a0a50d75d15daf)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:25:04.872977 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dccf7f-d57a-4237-ac34-2a48fc6b9025 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:25:04.920903 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-32dccf7f-d57a-4237-ac34-2a48fc6b9025 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4037ed70420a475cb2a0a50d75d15daf', 'request_id': 'req-32dccf7f-d57a-4237-ac34-2a48fc6b9025'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:25:04.922532 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:25:04.929182 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-32dccf7f-d57a-4237-ac34-2a48fc6b9025 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4037ed70420a475cb2a0a50d75d15daf', 'request_id': 'req-32dccf7f-d57a-4237-ac34-2a48fc6b9025'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:25:04.936621 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-32dccf7f-d57a-4237-ac34-2a48fc6b9025 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4037ed70420a475cb2a0a50d75d15daf', 'request_id': 'req-32dccf7f-d57a-4237-ac34-2a48fc6b9025'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:25:04.946960 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-32dccf7f-d57a-4237-ac34-2a48fc6b9025 admin admin] The token cache is being invalidate because project 4037ed70420a475cb2a0a50d75d15daf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81661) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:25:04.947244 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-32dccf7f-d57a-4237-ac34-2a48fc6b9025 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:25:04.947867 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-32dccf7f-d57a-4237-ac34-2a48fc6b9025 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:25:04.949342 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1453/2922] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:25:04 2022] DELETE /identity/v3/projects/4037ed70420a475cb2a0a50d75d15daf => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:25:05.701326 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ac21a-84db-4e10-9f8d-8750d436a1ae admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:25:05.702829 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-b58ac21a-84db-4e10-9f8d-8750d436a1ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:25:05.761463 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ac21a-84db-4e10-9f8d-8750d436a1ae admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:25:05.809560 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78927b9-dc46-43d9-8f6a-9df1db8c68c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5dbbcc3dbcc745458e97b31c6781ce70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:25:05.810159 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78927b9-dc46-43d9-8f6a-9df1db8c68c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:25:05.810437 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78927b9-dc46-43d9-8f6a-9df1db8c68c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:25:05.810709 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78927b9-dc46-43d9-8f6a-9df1db8c68c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:25:05.812480 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-d78927b9-dc46-43d9-8f6a-9df1db8c68c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:25:05.873917 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78927b9-dc46-43d9-8f6a-9df1db8c68c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:25:05.875347 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78927b9-dc46-43d9-8f6a-9df1db8c68c0 service neutron] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:25:05.885311 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1470/2923] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:25:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 01:25:11.903532 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dccf7f-d57a-4237-ac34-2a48fc6b9025 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:25:11.905132 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-32dccf7f-d57a-4237-ac34-2a48fc6b9025 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:25:11.918495 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dccf7f-d57a-4237-ac34-2a48fc6b9025 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:25:11.923262 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7825d8df-bede-4275-94aa-d56eb4dbf90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:25:11.923890 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825d8df-bede-4275-94aa-d56eb4dbf90b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:25:11.924206 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825d8df-bede-4275-94aa-d56eb4dbf90b admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:25:11.924484 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825d8df-bede-4275-94aa-d56eb4dbf90b admin admin] PATH_INFO: `/v3/users/744fe07ea5644157bdbf11d81adbab89` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:25:11.925605 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7825d8df-bede-4275-94aa-d56eb4dbf90b admin admin] RBAC: Authorizing `identity:delete_user(user_id=744fe07ea5644157bdbf11d81adbab89)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:25:11.927581 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7825d8df-bede-4275-94aa-d56eb4dbf90b admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:25:11.975790 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-7825d8df-bede-4275-94aa-d56eb4dbf90b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '744fe07ea5644157bdbf11d81adbab89'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:25:11.987620 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-7825d8df-bede-4275-94aa-d56eb4dbf90b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '744fe07ea5644157bdbf11d81adbab89'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:25:11.994760 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-7825d8df-bede-4275-94aa-d56eb4dbf90b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '744fe07ea5644157bdbf11d81adbab89'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:25:11.995444 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-7825d8df-bede-4275-94aa-d56eb4dbf90b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '744fe07ea5644157bdbf11d81adbab89'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:25:12.004543 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-7825d8df-bede-4275-94aa-d56eb4dbf90b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '744fe07ea5644157bdbf11d81adbab89'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:25:12.010981 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1454/2924] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:25:11 2022] DELETE /identity/v3/users/744fe07ea5644157bdbf11d81adbab89 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:25:12.187594 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78927b9-dc46-43d9-8f6a-9df1db8c68c0 service neutron] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:25:12.200317 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78927b9-dc46-43d9-8f6a-9df1db8c68c0 service neutron] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:25:12.258991 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:25:12.266620 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e903ce-e9b2-454f-855e-12d593eadcca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:25:12.267541 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e903ce-e9b2-454f-855e-12d593eadcca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:25:12.267999 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e903ce-e9b2-454f-855e-12d593eadcca admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:25:12.268474 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e903ce-e9b2-454f-855e-12d593eadcca admin admin] PATH_INFO: `/v3/projects/4221b10efb3d4bb799d820e985a48651` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:25:12.270337 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e903ce-e9b2-454f-855e-12d593eadcca admin admin] RBAC: Authorizing `identity:delete_project(project_id=4221b10efb3d4bb799d820e985a48651)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:25:12.272392 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e903ce-e9b2-454f-855e-12d593eadcca admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:25:12.320430 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-f5e903ce-e9b2-454f-855e-12d593eadcca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4221b10efb3d4bb799d820e985a48651', 'request_id': 'req-f5e903ce-e9b2-454f-855e-12d593eadcca'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:25:12.326528 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-f5e903ce-e9b2-454f-855e-12d593eadcca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4221b10efb3d4bb799d820e985a48651', 'request_id': 'req-f5e903ce-e9b2-454f-855e-12d593eadcca'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:25:12.332880 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-f5e903ce-e9b2-454f-855e-12d593eadcca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4221b10efb3d4bb799d820e985a48651', 'request_id': 'req-f5e903ce-e9b2-454f-855e-12d593eadcca'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:25:12.345342 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-f5e903ce-e9b2-454f-855e-12d593eadcca admin admin] The token cache is being invalidate because project 4221b10efb3d4bb799d820e985a48651 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:25:12.345649 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-f5e903ce-e9b2-454f-855e-12d593eadcca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:25:12.346257 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-f5e903ce-e9b2-454f-855e-12d593eadcca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:25:12.348546 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1471/2925] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:25:12 2022] DELETE /identity/v3/projects/4221b10efb3d4bb799d820e985a48651 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:25:12.352747 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7825d8df-bede-4275-94aa-d56eb4dbf90b admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:25:12.354277 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-7825d8df-bede-4275-94aa-d56eb4dbf90b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:25:12.367512 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7825d8df-bede-4275-94aa-d56eb4dbf90b admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:25:12.423371 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2713e967-f22c-4ca2-80a8-095bb8e7dec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:25:12.424019 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2713e967-f22c-4ca2-80a8-095bb8e7dec9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:25:12.424324 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2713e967-f22c-4ca2-80a8-095bb8e7dec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:25:12.424593 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2713e967-f22c-4ca2-80a8-095bb8e7dec9 admin admin] PATH_INFO: `/v3/users/14299221fc9d4ef1a202b08cdfd51503` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:25:12.425702 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2713e967-f22c-4ca2-80a8-095bb8e7dec9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14299221fc9d4ef1a202b08cdfd51503)` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:25:12.427620 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2713e967-f22c-4ca2-80a8-095bb8e7dec9 admin admin] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:25:12.473443 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-2713e967-f22c-4ca2-80a8-095bb8e7dec9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14299221fc9d4ef1a202b08cdfd51503'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:25:12.477837 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:INSERT=2,keystone:DELETE=20,keystone:SELECT=78 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:25:12.488225 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-2713e967-f22c-4ca2-80a8-095bb8e7dec9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14299221fc9d4ef1a202b08cdfd51503'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:25:12.495440 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-2713e967-f22c-4ca2-80a8-095bb8e7dec9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14299221fc9d4ef1a202b08cdfd51503'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:25:12.496150 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-2713e967-f22c-4ca2-80a8-095bb8e7dec9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14299221fc9d4ef1a202b08cdfd51503'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:25:12.503239 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.notifications [None req-2713e967-f22c-4ca2-80a8-095bb8e7dec9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14299221fc9d4ef1a202b08cdfd51503'} {{(pid=81661) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:25:12.509678 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1455/2926] 10.222.0.42 () {58 vars in 1224 bytes} [Thu Sep 8 01:25:12 2022] DELETE /identity/v3/users/14299221fc9d4ef1a202b08cdfd51503 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:25:12.681007 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e903ce-e9b2-454f-855e-12d593eadcca admin admin] Authenticating user token {{(pid=81660) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:25:12.682441 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.fernet_utils [None req-f5e903ce-e9b2-454f-855e-12d593eadcca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81660) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:25:12.697242 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e903ce-e9b2-454f-855e-12d593eadcca admin admin] Validating token access rules against request {{(pid=81660) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:25:12.757480 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e88849-ce36-497a-92de-680c5cf7da3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f97eb5e7894117a81b5bd04a11488e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc3cae79b39a4ab893f53897bb099478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81660) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:25:12.758363 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e88849-ce36-497a-92de-680c5cf7da3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:25:12.758641 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e88849-ce36-497a-92de-680c5cf7da3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:25:12.758910 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e88849-ce36-497a-92de-680c5cf7da3e admin admin] PATH_INFO: `/v3/projects/5905339d520b40308156ab0533803bde` {{(pid=81660) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:25:12.760010 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e88849-ce36-497a-92de-680c5cf7da3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5905339d520b40308156ab0533803bde)` {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:25:12.761938 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e88849-ce36-497a-92de-680c5cf7da3e admin admin] RBAC: Authorization granted {{(pid=81660) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:25:12.809656 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-a7e88849-ce36-497a-92de-680c5cf7da3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5905339d520b40308156ab0533803bde', 'request_id': 'req-a7e88849-ce36-497a-92de-680c5cf7da3e'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:25:12.815725 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-a7e88849-ce36-497a-92de-680c5cf7da3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5905339d520b40308156ab0533803bde', 'request_id': 'req-a7e88849-ce36-497a-92de-680c5cf7da3e'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:25:12.822083 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-a7e88849-ce36-497a-92de-680c5cf7da3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5905339d520b40308156ab0533803bde', 'request_id': 'req-a7e88849-ce36-497a-92de-680c5cf7da3e'} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:25:12.834696 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-a7e88849-ce36-497a-92de-680c5cf7da3e admin admin] The token cache is being invalidate because project 5905339d520b40308156ab0533803bde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81660) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 01:25:12.834986 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-a7e88849-ce36-497a-92de-680c5cf7da3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:25:12.835605 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG keystone.notifications [None req-a7e88849-ce36-497a-92de-680c5cf7da3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81660) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 01:25:12.837107 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: [pid: 81660|app: 0|req: 1472/2927] 10.222.0.42 () {58 vars in 1233 bytes} [Thu Sep 8 01:25:12 2022] DELETE /identity/v3/projects/5905339d520b40308156ab0533803bde => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 01:25:22.505931 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:DELETE=3,keystone:SELECT=10,keystone:INSERT=1 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:25:22.831214 dv-cinder-847733-3-iscsi devstack@keystone.service[81660]: DEBUG dbcounter [-] [81660] Writing DB stats keystone:SELECT=76,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81660) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 01:26:32.821259 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2713e967-f22c-4ca2-80a8-095bb8e7dec9 admin admin] Authenticating user token {{(pid=81661) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 01:26:32.822685 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-2713e967-f22c-4ca2-80a8-095bb8e7dec9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:26:32.895042 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2713e967-f22c-4ca2-80a8-095bb8e7dec9 admin admin] Validating token access rules against request {{(pid=81661) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 01:26:32.956601 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06814941-2353-43f3-8e7b-d353179e0b54 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '301d30bbd3de4cc69e96bb12729442c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78ad2a8fb5264fb4af935e039d28c8de', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81661) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 01:26:32.957209 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06814941-2353-43f3-8e7b-d353179e0b54 service placement] REQUEST_METHOD: `GET` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 01:26:32.957487 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06814941-2353-43f3-8e7b-d353179e0b54 service placement] SCRIPT_NAME: `/identity` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 01:26:32.957758 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06814941-2353-43f3-8e7b-d353179e0b54 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81661) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 01:26:32.959478 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.fernet_utils [None req-06814941-2353-43f3-8e7b-d353179e0b54 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81661) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 01:26:32.970822 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06814941-2353-43f3-8e7b-d353179e0b54 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 01:26:32.972432 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06814941-2353-43f3-8e7b-d353179e0b54 service placement] RBAC: Authorization granted {{(pid=81661) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 01:26:32.982480 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: [pid: 81661|app: 0|req: 1456/2928] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Sep 8 01:26:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 01:26:42.977990 dv-cinder-847733-3-iscsi devstack@keystone.service[81661]: DEBUG dbcounter [-] [81661] Writing DB stats keystone:SELECT=58 {{(pid=81661) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}